Faster title and abstract screening? Evaluating Abstrackr, a semi-automated online screening program for systematic reviewers

Chatbots for conducting systematic reviews in pediatric dentistry

OBJECTIVES

The performance of chatbots for discrete steps of a systematic review (SR) on artificial intelligence (AI) in pediatric dentistry was evaluated.

METHODS

Two chatbots (ChatGPT4/Gemini) and two non-expert reviewers were compared against two experts in a SR on AI in pediatric dentistry. Five tasks: (1) formulating a PICO question, (2) developing search queries for eight databases, (3) screening studies, (4) extracting data, and (5) assessing the risk of bias (RoB) were assessed. Chatbots and non-experts received identical prompts, with experts providing the reference standard. Performance was measured using accuracy, precision, sensitivity, specificity, and F1-score for search and screening tasks, Cohen's Kappa for risk of bias assessment, and a modified Global Quality Score (1-5) for PICO question formulation and data extraction quality. Statistical comparisons were performed using Kruskal-Wallis and Dunn's post-hoc tests.

RESULTS

In PICO formulation, ChatGPT outperformed Gemini slightly, while non-experts scored the lowest. Experts identified 1261 records, compared to 569 (ChatGPT), 285 (Gemini), and 722 (non-experts). Screening showed chatbots having 90 % sensitivity, >60 % specificity, <25 % precision, and F1-scores <40 %, versus non-experts' 84 % sensitivity, 91 % specificity, and 39 % F1-score, respectively. For data extraction, ChatGPT yielded a (mean±standard deviation) score of 31.6 ± 12.3 (max. was 45), Gemini 29.2 ± 12.3, and non-experts 30.4 ± 11.3, respectively. For RoB, the agreement with experts was 49.4 % for ChatGPT, 51.2 % for Gemini 48.8 % for non-experts (p > 0.05).

CONCLUSION

Chatbots could enhance SR efficiency, particularly for the study screening and data extraction steps. Human oversight remains critical for ensuring accuracy and completeness.

CLINICAL SIGNIFICANCE

Chatbots can streamline SR tasks like screening and data extraction, potentially accelerating evidence synthesis, though human oversight remains needed for reliability. The applicability of chatbots for SR steps was found dependent on the specific step, indicating reviewers need to make informed choices when employing chatbots for this purpose.

Streamlining systematic reviews with large language models using prompt engineering and retrieval augmented generation

BACKGROUND

Systematic reviews (SRs) are essential to formulate evidence-based guidelines but require time-consuming and costly literature screening. Large Language Models (LLMs) can be a powerful tool to expedite SRs.

METHODS

We conducted a comparative study to evaluate the performance of a commercial tool, Rayyan, and an in-house LLM-based system in automating the screening of a completed SR on Vitamin D and falls. The SR retrieved 14,439 articles, and Rayyan was trained with 2,000 manually screened articles to categorize the rest as most likely to exclude/include, likely to exclude/include and undecided. We analyzed Rayyan's title/abstract screening performance using different inclusion thresholds. For the LLM, we used prompt engineering for title/abstract screening and Retrieval-Augmented Generation (RAG) for full-text screening. We evaluated performance using article exclusion rate (AER), false negative rate (FNR), specificity, positive predictive value (PPV), and negative predictive value (NPV). Additionally, we compared the time required to complete screening steps of the SR using both approaches against the manual screening method.

RESULTS

Using Rayyan, including considered as undecided or likely to include for title/abstract screening resulted in an AER of 72.1% and an FNR of 5%. The total estimated screening time, including manual review of articles flagged by Rayyan, was 54.7 hours. Lowering the Rayyan threshold to 'likely to exclude' reduced the FNR to 0% and the AER to 50.7%, but increased the screening time to 81.3 h. Using the LLM system, after title/abstract and full-text screening, 78 articles remained for manual review, including all 20 identified by traditional methods. The LLM achieved an AER of 99.5%, specificity of 99.6%, PPV of 25.6%, and NPV of 100%, with a total screening time of 25.5 h, including manual review of the 78 articles, reducing the manual screening time by 95.5%.

CONCLUSIONS

The LLM-based system significantly enhances SR efficiency, compared to manual methods and Rayyan while maintaining low FNR.

Artificial Intelligence Software to Accelerate Screening for Living Systematic Reviews

Systematic and meta-analytic reviews provide gold-standard evidence but are static and outdate quickly. Here we provide performance data on a new software platform, LitQuest, that uses artificial intelligence technologies to (1) accelerate screening of titles and abstracts from library literature searches, and (2) provide a software solution for enabling living systematic reviews by maintaining a saved AI algorithm for updated searches. Performance testing was based on LitQuest data from seven systematic reviews. LitQuest efficiency was estimated as the proportion (%) of the total yield of an initial literature search (titles/abstracts) that needed human screening prior to reaching the in-built stop threshold. LitQuest algorithm performance was measured as work saved over sampling (WSS) for a certain recall. LitQuest accuracy was estimated as the proportion of incorrectly classified papers in the rejected pool, as determined by two independent human raters. On average, around 36% of the total yield of a literature search needed to be human screened prior to reaching the stop-point. However, this ranged from 22 to 53% depending on the complexity of language structure across papers included in specific reviews. Accuracy was 99% at an interrater reliability of 95%, and 0% of titles/abstracts were incorrectly assigned. Findings suggest that LitQuest can be a cost-effective and time-efficient solution to supporting living systematic reviews, particularly for rapidly developing areas of science. Further development of LitQuest is planned, including facilitated full-text data extraction and community-of-practice access to living systematic review findings.

Systematic review of AI/ML applications in multi-domain robotic rehabilitation: trends, gaps, and future directions

Robotic technology is expected to transform rehabilitation settings, by providing precise, repetitive, and task-specific interventions, thereby potentially improving patients' clinical outcomes. Artificial intelligence (AI) and machine learning (ML) have been widely applied in different areas to support robotic rehabilitation, from controlling robot movements to real-time patient assessment. To provide an overview of the current landscape and the impact of AI/ML use in robotics rehabilitation, we performed a systematic review focusing on the use of AI and robotics in rehabilitation from a broad perspective, encompassing different pathologies and body districts, and considering both motor and neurocognitive rehabilitation. We searched the Scopus and IEEE Xplore databases, focusing on the studies involving human participants. After article retrieval, a tagging phase was carried out to devise a comprehensive and easily-interpretable taxonomy: its categories include the aim of the AI/ML within the rehabilitation system, the type of algorithms used, and the location of robots and sensors. The 201 selected articles span multiple domains and diverse aims, such as movement classification, trajectory prediction, and patient evaluation, demonstrating the potential of ML to revolutionize personalized therapy and improve patient engagement. ML is reported as highly effective in predicting movement intentions, assessing clinical outcomes, and detecting compensatory movements, providing insights into the future of personalized rehabilitation interventions. Our analysis also reveals pitfalls in the current use of AI/ML in this area, such as potential explainability issues and poor generalization ability when these systems are applied in real-world settings.

Digital Tools to Support the Systematic Review Process: An Introduction

BACKGROUND

The introduction of systematic reviews in medicine has prompted a paradigm shift in employing evidence for decision-making across various fields. Its methodology involves structured comparisons, critical appraisals, and pooled data analysis to inform decision-making. The process itself is resource-intensive and time-consuming which can impede the timely incorporation of the latest evidence into clinical practice.

AIM

This article introduces digital tools designed to enhance systematic review processes, emphasizing their functionality, availability, and independent validation in peer-reviewed literature.

METHODS

We discuss digital evidence synthesis tools for systematic reviews, identifying tools for all review processes, tools for search strategy development, reference management, study selection, data extraction, and critical appraisal. Emphasis is on validated, functional tools with independently published method evaluations.

RESULTS

Tools like EPPI-Reviewer, Covidence, DistillerSR, and JBI-SUMARI provide comprehensive support for systematic reviews. Additional tools cater to evidence search (e.g., PubMed PICO, Trialstreamer), reference management (e.g., Mendeley), prioritization in study selection (e.g., Abstrackr, EPPI-Reviewer, SWIFT-ActiveScreener), and risk bias assessment (e.g., RobotReviewer). Machine learning and AI integration facilitate workflow efficiency but require end-user informed evaluation for their adoption.

CONCLUSION

The development of digital tools, particularly those incorporating AI, represents a significant advancement in systematic review methodology. These tools not only support the systematic review process but also have the potential to improve the timeliness and quality of evidence available for decision-making. The findings are relevant to clinicians, researchers, and those involved in the production or support of systematic reviews, with broader applicability to other research methods.

Pre-diagnostic circulating metabolomics and prostate cancer risk: A systematic review and meta-analysis

Background

Metabolomic dysregulation contributes to prostate cancer (PCa) pathogenesis, and studies suggest that circulating metabolites have strong clinical potential to act as biomarkers. However, evidence of circulating metabolite associations has not been quantitively aggregated.

Methods

Systematic searches were performed in PubMed and Embase (October 17th, 2024) to identify pre-diagnostic untargeted serum metabolomic studies of PCa risk. After harmonizing metabolite names across studies, restricted maximum likelihood was used to conduct meta-analyses to quantify associations between metabolites and risk of overall PCa, low- to intermediate-risk PCa, high- to very high-risk PCa and lethal PCa, as defined by the NCCN. Statistical significance was defined as FDR-adjusted P<0.05. Enrichment analyses were conducted on significant metabolites to identify biologically relevant pathways. Correlation of effect estimates between PCa outcomes was assessed via Pearson correlation.

Results

We identified 12 untargeted pre-diagnostic circulating metabolomic studies in a systematic review and meta-analyzed associations between up to 408 metabolites with four PCa outcomes. Three, eleven and nineteen metabolites were significantly associated with risk of overall, high/very high-risk and lethal PCa, respectively. Metabolites associated with high/very high-risk PCa were significantly enriched for lipids. Limited evidence of correlation between metabolite effects across outcomes was identified, highlighting potentially unique metabolite drivers of high-risk and lethal PCa. Follow-up analyses found that 13 of the significant metabolites were drug and/or dietary modifiable.

Conclusions

These findings suggest the strong potential for metabolites to inform risk of lethal PCa, which could inform risk-stratified screening strategies and facilitate the identification of targets for PCa prevention.

Substituting confidence for competence in health literacy: a review of studies, citations, and trial registrations

Patient health literacy is crucial for effective patient-physician communication, and interventions targeting health literacy can use measures based on either actual performance (competence) or self-ratings (confidence). This paper analyzed the development of these measures through three studies. Study 1 reviewed articles describing the development of novel measures; Study 2 examined the citations of these studies, and Study 3 evaluated data from clinical trials registries. The literature search was conducted from 14 April 2023 to 27 April 2023. PubMed was used as the main database in which studies on health literacy measures were searched for the systematic review (Study 1). We then used Google Scholar and the OpenCitations database to describe citation patterns of the included health literacy measures (Study 2). Finally, we evaluated confidence- or competence-based health literacy measures by extracting and analyzing trial data from ClinicalTrials.gov (Study 3). Our review included 55 health literacy measures, among which 23 (42%) were competence-based, 28 (51%) confidence-based, and 4 (7%) assessed both. Recent trends show a shift toward developing more confidence-based measures and a decline in creating new competence-based measures. Confidence-based measures were increasingly cited, whereas citations for competence-based measures have plateaued. Lastly, our findings showed a steady increase in the use of confidence-based measures in recent clinical trials and a decrease in the use of competence-based measures when controlling for sample size. This shift may be problematic because confidence-based measures do not improve our limited knowledge about patients' actual ability to meet demands of shared decision-making, especially regarding new technologies like artificial intelligence in healthcare.

Development and Validation of a Literature Screening Tool: Few-Shot Learning Approach in Systematic Reviews

BACKGROUND

Systematic reviews (SRs) are considered the highest level of evidence, but their rigorous literature screening process can be time-consuming and resource-intensive. This is particularly challenging given the rapid pace of medical advancements, which can quickly make SRs outdated. Few-shot learning (FSL), a machine learning approach that learns effectively from limited data, offers a potential solution to streamline this process. Sentence-bidirectional encoder representations from transformers (S-BERT) are particularly promising for identifying relevant studies with fewer examples.

OBJECTIVE

This study aimed to develop a model framework using FSL to efficiently screen and select relevant studies for inclusion in SRs, aiming to reduce workload while maintaining high recall rates.

METHODS

We developed and validated the FSL model framework using 9 previously published SR projects (2016-2018). The framework used S-BERT with titles and abstracts as input data. Key evaluation metrics, including workload reduction, cosine similarity score, and the number needed to screen at 100% recall, were estimated to determine the optimal number of eligible studies for model training. A prospective evaluation phase involving 4 ongoing SRs was then conducted. Study selection by FSL and a secondary reviewer were compared with the principal reviewer (considered the gold standard) to estimate the false negative rate.

RESULTS

Model development suggested an optimal range of 4-12 eligible studies for FSL training. Using 4-6 eligible studies during model development resulted in similarity thresholds for 100% recall, ranging from 0.432 to 0.636, corresponding to a workload reduction of 51.11% (95% CI 46.36-55.86) to 97.67% (95% CI 96.76-98.58). The prospective evaluation of 4 SRs aimed for a 50% workload reduction, yielding numbers needed to screen 497 to 1035 out of 995 to 2070 studies. The false negative rate ranged from 1.87% to 12.20% for the FSL model and from 5% to 56.48% for the second reviewer compared with the principal reviewer.

CONCLUSIONS

Our FSL framework demonstrates the potential for reducing workload in SR screening by over 50%. However, the model did not achieve 100% recall at this threshold, highlighting the potential for omitting eligible studies. Future work should focus on developing a web application to implement the FSL framework, making it accessible to researchers.

Effects of Varying Protein Amounts and Types on Diet-Induced Thermogenesis: A Systematic Review and Meta-Analysis

Protein is the most thermogenic macronutrient, but it is unclear how different amounts and types of protein impact diet-induced thermogenesis (DIT). The purpose of this meta-analysis was to compare the impact of isocaloric meals/diets containing different amounts or types of protein on energy metabolism. Databases were searched in June 2024 for studies that compare DIT or total daily energy expenditure (TDEE) in response to isocaloric acute meals or longer-term diets containing different amounts or types of protein. After identifying 3894 records, 52 studies were included. Standardized mean difference (SMD) estimates and 95% confidence intervals (CIs) were calculated for each outcome. In acute studies, intake of higher compared with lower-protein meals resulted in greater DIT (SMD: 0.45; 95% CI: 0.26, 0.65; P < 0.001) and TDEE (SMD: 0.52; 95% CI: 0.30, 0.73; P < 0.001). Notably, the subgroup analysis indicated that this effect on DIT was statistically significant for studies involving participants with normal weight but not overweight/obesity, although it is not clear if this finding was a true effect or because of study design characteristics. In chronic studies (ranging from 4 d to 1 y), intake of higher compared with lower-protein diets resulted in greater TDEE (SMD: 0.29; 95% CI: 0.10, 0.48; P = 0.003) and resting energy expenditure (SMD: 0.18; 95% CI: 0.01, 0.35; P = 0.039), but no differences in DIT (SMD: 0.10; 95% CI: -0.08, 0.28; P = 0.27). There was no evidence that different types of protein impacted energy metabolism. Higher protein meals/diets increase components of energy expenditure. This trial was registered at the International Prospective Register of Systematic Reviews (https://www.crd.york.ac.uk/prospero; PROSPERO 2023) as CRD42023389642.

Automation tools to support undertaking scoping reviews

OBJECTIVE

This paper describes several automation tools and software that can be considered during evidence synthesis projects and provides guidance for their integration in the conduct of scoping reviews.

STUDY DESIGN AND SETTING

The guidance presented in this work is adapted from the results of a scoping review and consultations with the JBI Scoping Review Methodology group.

RESULTS

This paper describes several reliable, validated automation tools and software that can be used to enhance the conduct of scoping reviews. Developments in the automation of systematic reviews, and more recently scoping reviews, are continuously evolving. We detail several helpful tools in order of the key steps recommended by the JBI's methodological guidance for undertaking scoping reviews including team establishment, protocol development, searching, de-duplication, screening titles and abstracts, data extraction, data charting, and report writing. While we include several reliable tools and software that can be used for the automation of scoping reviews, there are some limitations to the tools mentioned. For example, some are available in English only and their lack of integration with other tools results in limited interoperability.

CONCLUSION

This paper highlighted several useful automation tools and software programs to use in undertaking each step of a scoping review. This guidance has the potential to inform collaborative efforts aiming at the development of evidence informed, integrated automation tools and software packages for enhancing the conduct of high-quality scoping reviews.

Leaking in Intimate Partner Homicide: A Systematic Review

Intimate partner homicides (IPH) are serious offenses by a heterogeneous group of offenders with diverse risk factors that are too unspecific for the successful prediction of an offense. Recent research suggested several warning signs that may precede IPH and enhance its prevention, but little is still known about "leaking." Leaking comprises all offense-related statements, behaviors, or actions that express the perpetrator's thoughts, fantasies, ideas, interests, feelings, intentions, plans, or positive evaluations of an own violent act or previous similar offenses prior to the own attack. This review aims to identify the forms, recipients, and media of leaking as well as potential subgroup differences in cases of IPH. We identified 47 relevant publications via a systematic search of eight databases and additional methods. We included publications that did not explicitly use the term, but described behaviors that could be interpreted as leaking. Up to now, leaking has not been systematically researched in cases of IPH. Nevertheless, publications described several behaviors that are in line with our definition of leaking and were categorized into five broader categories: (a) homicide announcements, (b) previous severe acts of violence, (c) suicidal behavior, (d) planning activities, and (e) interest in similar offenses/offenders. Information on recipients and media as well as subgroup differences was sparse. Leaking is relevant in IPH, but more systematic research is needed to understand its potential role in future risk analyses procedures and prevention of IPH.

Systematic Review of Gaming and Neuropsychological Assessment of Social Cognition

Playing video games is associated with cognitive changes and possibly psychosocial difficulties. Problematic gaming occurs upon the loss of control over videogame playing; gaming disorder is considered a behavioral addiction in the 11th version of the International Classification of Diseases. Models used to understand behavioral addictions include cognition as an essential factor in the development, maintenance, and relapse of addiction. Nevertheless, some aspects of cognition, such as social cognition, remain underexplored, despite evidence of alterations in cognitive and social function among patients with problematic gaming. This review aimed to describe the current understanding of social cognition in individuals exposed to videogames. We included all studies assessing social cognition in participants of any age with a wide range of exposure to video games (from simple use of video games (such as at least two exposures) to problematic gaming, defined according to the included study). This wide range of exposure allowed us to explore the whole process from repeated exposure to addiction. We included only studies that used neuropsychological tasks to assess social cognition. Patient-reported outcomes that could be biased by subjective self-report data were not included. The search was conducted from inception to January 2022 in three databases (PubMed, PsycINFO, and Web of Science). The systematic search identified 39 studies that assessed facial emotion processing, empathy, theory of mind, social decision-making, aggressive behavior, and moral competence. In general, results have been mixed, and a number of questions remain unanswered. Nevertheless, several studies showed cerebral changes when processing facial emotion that were linked with problematic gaming, while no link was obtained between nonproblematic gaming and empathy alterations. The influences of cooperation patterns, theory of mind, moral competence, and gaming frequency were highlighted. Finally, there was substantial heterogeneity in the population assessed and the methods used.

Cutting-Edge Methodological Guidance for Authors in Conducting the Systematic Review and Meta-Analysis

The landscape of systematic reviews and meta-analyses (SRMA) in biomedicine has expanded exponentially, driven by the growing demand for evidence-based healthcare decision-making. However, the rapid increase of SRMAs has often outpaced the development of rigorous methodological standards, resulting in variability in quality and potentially limiting their effectiveness in informing healthcare practices. This gap highlights the critical need for advanced methodological guidance to enhance the quality and impact of SRMAs. Our contribution aims to provide comprehensive methodological direction for authors to conduct robust SRMAs. By effectively integrating qualitative and quantitative evidence, SRMAs can address complex healthcare questions more thoroughly than traditional reviews. Furthermore, these step-by-step guidelines will help researchers to address the challenges of synthesizing diverse types of evidence, thereby improving the rigor, relevance, and applicability of their findings in healthcare decision-making processes.

Human-Comparable Sensitivity of Large Language Models in Identifying Eligible Studies Through Title and Abstract Screening: 3-Layer Strategy Using GPT-3.5 and GPT-4 for Systematic Reviews

BACKGROUND

The screening process for systematic reviews is resource-intensive. Although previous machine learning solutions have reported reductions in workload, they risked excluding relevant papers.

OBJECTIVE

We evaluated the performance of a 3-layer screening method using GPT-3.5 and GPT-4 to streamline the title and abstract-screening process for systematic reviews. Our goal is to develop a screening method that maximizes sensitivity for identifying relevant records.

METHODS

We conducted screenings on 2 of our previous systematic reviews related to the treatment of bipolar disorder, with 1381 records from the first review and 3146 from the second. Screenings were conducted using GPT-3.5 (gpt-3.5-turbo-0125) and GPT-4 (gpt-4-0125-preview) across three layers: (1) research design, (2) target patients, and (3) interventions and controls. The 3-layer screening was conducted using prompts tailored to each study. During this process, information extraction according to each study's inclusion criteria and optimization for screening were carried out using a GPT-4-based flow without manual adjustments. Records were evaluated at each layer, and those meeting the inclusion criteria at all layers were subsequently judged as included.

RESULTS

On each layer, both GPT-3.5 and GPT-4 were able to process about 110 records per minute, and the total time required for screening the first and second studies was approximately 1 hour and 2 hours, respectively. In the first study, the sensitivities/specificities of the GPT-3.5 and GPT-4 were 0.900/0.709 and 0.806/0.996, respectively. Both screenings by GPT-3.5 and GPT-4 judged all 6 records used for the meta-analysis as included. In the second study, the sensitivities/specificities of the GPT-3.5 and GPT-4 were 0.958/0.116 and 0.875/0.855, respectively. The sensitivities for the relevant records align with those of human evaluators: 0.867-1.000 for the first study and 0.776-0.979 for the second study. Both screenings by GPT-3.5 and GPT-4 judged all 9 records used for the meta-analysis as included. After accounting for justifiably excluded records by GPT-4, the sensitivities/specificities of the GPT-4 screening were 0.962/0.996 in the first study and 0.943/0.855 in the second study. Further investigation indicated that the cases incorrectly excluded by GPT-3.5 were due to a lack of domain knowledge, while the cases incorrectly excluded by GPT-4 were due to misinterpretations of the inclusion criteria.

CONCLUSIONS

Our 3-layer screening method with GPT-4 demonstrated acceptable level of sensitivity and specificity that supports its practical application in systematic review screenings. Future research should aim to generalize this approach and explore its effectiveness in diverse settings, both medical and nonmedical, to fully establish its use and operational feasibility.

Rapid review: A review of methods and recommendations based on current evidence

Rapid review (RR) could accelerate the traditional systematic review (SR) process by simplifying or omitting steps using various shortcuts. With the increasing popularity of RR, numerous shortcuts had emerged, but there was no consensus on how to choose the most appropriate ones. This study conducted a literature search in PubMed from inception to December 21, 2023, using terms such as "rapid review" "rapid assessment" "rapid systematic review" and "rapid evaluation". We also scanned the reference lists and performed citation tracking of included impact studies to obtain more included studies. We conducted a narrative synthesis of all RR approaches, shortcuts and studies assessing their effectiveness at each stage of RRs. Based on the current evidence, we provided recommendations on utilizing certain shortcuts in RRs. Ultimately, we identified 185 studies focusing on summarizing RR approaches and shortcuts, or evaluating their impact. There was relatively sufficient evidence to support the use of the following shortcuts in RRs: limiting studies to those published in English-language; conducting abbreviated database searches (e.g., only searching PubMed/MEDLINE, Embase, and CENTRAL); omitting retrieval of grey literature; restricting the search timeframe to the recent 20 years for medical intervention and the recent 15 years for reviewing diagnostic test accuracy; conducting a single screening by an experienced screener. To some extent, the above shortcuts were also applicable to SRs. This study provided a reference for future RR researchers in selecting shortcuts, and it also presented a potential research topic for methodologists.

Broccoli Consumption and Risk of Cancer: An Updated Systematic Review and Meta-Analysis of Observational Studies

BACKGROUND

The scientific literature has reported an inverse association between broccoli consumption and the risk of suffering from several types of cancer; however, the results were not entirely consistent across studies. A systematic review and meta-analysis of observational studies were conducted to determine the association between broccoli consumption and cancer risk with the aim of clarifying the beneficial biological effects of broccoli consumption on cancer.

METHODS

PubMed/MEDLINE, Web of Science, Scopus, Cochrane Library (CENTRAL), and Epistemonikos databases were searched to identify all published papers that evaluate the impact of broccoli consumption on the risk of cancer. Citation chasing of included studies was conducted as a complementary search strategy. The risk of bias in individual studies was assessed using the Newcastle-Ottawa Scale. A random-effects model meta-analysis was employed to quantitatively synthesize results, with the I2 index used to assess heterogeneity.

RESULTS

Twenty-three case-control studies (n = 12,929 cases and 18,363 controls; n = 31,292 individuals) and 12 cohort studies (n = 699,482 individuals) were included in the meta-analysis. The results suggest an inverse association between broccoli consumption and the risk of cancer both in case-control studies (OR: 0.64, 95% CI from 0.58 to 0.70, p < 0.001; Q = 35.97, p = 0.072, I2 = 30.49%-moderate heterogeneity; τ2 = 0.016) and cohort studies (RR: 0.89, 95% CI from 0.82 to 0.96, p = 0.003; Q = 13.51, p = 0.333, I2 = 11.21%-low heterogeneity; τ2 = 0.002). Subgroup analysis suggested a potential benefit of broccoli consumption in site-specific cancers only in case-control studies.

CONCLUSIONS

In summary, the findings indicate that individuals suffering from some type of cancer consumed less broccoli, suggesting a protective biological effect of broccoli on cancer. More studies, especially cohort studies, are necessary to clarify the possible beneficial effect of broccoli on several types of cancer.

Assets and depression in U.S. adults during the COVID-19 pandemic: a systematic review

PURPOSE

Mental health is shaped by social and economic contexts, which were altered during the COVID-19 pandemic. No study has systematically reviewed the literature on the relation between different assets and depression during the COVID-19 pandemic.

METHODS

We conducted a systematic review of the literature on financial (e.g. income/savings), physical (e.g., home ownership), and social (e.g., marital status, educational attainment) assets and depression in U.S. adults. For each asset type, we created binary comparisons to report on the direction of the relationship and described if each study reported insignificant, positive, negative, or mixed associations.

RESULTS

Among the 41 articles identified, we found that income was the most studied asset (n=34), followed by education (n=25), marital status (n=18), home ownership (n=5), and savings (n=4). 88%, 100%, and 100% of articles reported a significant association of higher income, home ownership, and higher savings, respectively, with less depression. The association between marital status and education with depression was more nuanced: 72% (13 of 18) studies showed that unmarried persons had greater risk of depression than married or cohabitating persons and 52% (13 of 25) of studies reported no significant difference in depression across educational groups.

CONCLUSION

This work adds to the literature a deeper understanding of how different assets relate to depression. In the context of largescale traumatic events, policies that maintain and protect access to social, physical, and financial assets may help to protect mental health.

Methodological insights into ChatGPT's screening performance in systematic reviews

BACKGROUND

The screening process for systematic reviews and meta-analyses in medical research is a labor-intensive and time-consuming task. While machine learning and deep learning have been applied to facilitate this process, these methods often require training data and user annotation. This study aims to assess the efficacy of ChatGPT, a large language model based on the Generative Pretrained Transformers (GPT) architecture, in automating the screening process for systematic reviews in radiology without the need for training data.

METHODS

A prospective simulation study was conducted between May 2nd and 24th, 2023, comparing ChatGPT's performance in screening abstracts against that of general physicians (GPs). A total of 1198 abstracts across three subfields of radiology were evaluated. Metrics such as sensitivity, specificity, positive and negative predictive values (PPV and NPV), workload saving, and others were employed. Statistical analyses included the Kappa coefficient for inter-rater agreement, ROC curve plotting, AUC calculation, and bootstrapping for p-values and confidence intervals.

RESULTS

ChatGPT completed the screening process within an hour, while GPs took an average of 7-10 days. The AI model achieved a sensitivity of 95% and an NPV of 99%, slightly outperforming the GPs' sensitive consensus (i.e., including records if at least one person includes them). It also exhibited remarkably low false negative counts and high workload savings, ranging from 40 to 83%. However, ChatGPT had lower specificity and PPV compared to human raters. The average Kappa agreement between ChatGPT and other raters was 0.27.

CONCLUSIONS

ChatGPT shows promise in automating the article screening phase of systematic reviews, achieving high sensitivity and workload savings. While not entirely replacing human expertise, it could serve as an efficient first-line screening tool, particularly in reducing the burden on human resources. Further studies are needed to fine-tune its capabilities and validate its utility across different medical subfields.

Evaluating the efficacy of artificial intelligence tools for the automation of systematic reviews in cancer research: A systematic review

To evaluate the performance accuracy and workload savings of artificial intelligence (AI)-based automation tools in comparison with human reviewers in medical literature screening for systematic reviews (SR) of primary studies in cancer research in order to gain insights on improving the efficiency of producing SRs. Medline, Embase, the Cochrane Library, and PROSPERO databases were searched from inception to November 30, 2022. Then, forward and backward literature searches were completed, and the experts in this field including the authors of the articles included were contacted for a thorough grey literature search. This SR was registered on PROSPERO (CRD 42023384772). Among the 3947 studies obtained from search, five studies met the preplanned study selection criteria. These five studies evaluated four AI tools: Abstrackr (four studies), RobotAnalyst (one), EPPI-Reviewer (one), and DistillerSR (one). Without missing final included citations, Abstrackr eliminated 20%-88% of titles and abstracts (time saving of 7-86 hours) and 59% of the full-texts (62 h) from human review across four different cancer-related SRs. In comparison, RobotAnalyst (1% of titles and abstracts, 1 h), EPPI Review (38% of titles and abstracts, 58 h; 59% of full-texts, 62 h), DistillerSR (42% of titles and abstracts, 22 h) also provided similar or lower work savings for single cancer-related SRs. AI-based automation tools exhibited promising but varying levels of accuracy and efficiency during the screening process of medical literature for conducting SRs in the cancer field. Until further progress is made and thorough evaluations are conducted, AI tools should be utilized as supplementary aids rather than complete substitutes for human reviewers.

Systematic review using a spiral approach with machine learning

With the accelerating growth of the academic corpus, doubling every 9 years, machine learning is a promising avenue to make systematic review manageable. Though several notable advancements have already been made, the incorporation of machine learning is less than optimal, still relying on a sequential, staged process designed to accommodate a purely human approach, exemplified by PRISMA. Here, we test a spiral, alternating or oscillating approach, where full-text screening is done intermittently with title/abstract screening, which we examine in three datasets by simulation under 360 conditions comprised of different algorithmic classifiers, feature extractions, prioritization rules, data types, and information provided (e.g., title/abstract, full-text included). Overwhelmingly, the results favored a spiral processing approach with logistic regression, TF-IDF for vectorization, and maximum probability for prioritization. Results demonstrate up to a 90% improvement over traditional machine learning methodologies, especially for databases with fewer eligible articles. With these advancements, the screening component of most systematic reviews should remain functionally achievable for another one to two decades.

Meta-analysis shows no consistent evidence for senescence in ejaculate traits across animals

Male reproductive traits such as ejaculate size and quality, are expected to decline with advancing age due to senescence. It is however unclear whether this expectation is upheld across taxa. We perform a meta-analysis on 379 studies, to quantify the effects of advancing male age on ejaculate traits across 157 species of non-human animals. Contrary to predictions, we find no consistent pattern of age-dependent changes in ejaculate traits. This result partly reflects methodological limitations, such as studies sampling a low proportion of adult lifespan, or the inability of meta-analytical approaches to document non-linear ageing trajectories of ejaculate traits; which could potentially lead to an underestimation of senescence. Yet, we find taxon-specific differences in patterns of ejaculate senescence. For instance, older males produce less motile and slower sperm in ray-finned fishes, but larger ejaculates in insects, compared to younger males. Notably, lab rodents show senescence in most ejaculate traits measured. Our study challenges the notion of universal reproductive senescence, highlighting the need for controlled methodologies and a more nuanced understanding of reproductive senescence, cognisant of taxon-specific biology, experimental design, selection pressures, and life-history.

Interprofessional collaboration in primary care: what effect on patient health? A systematic literature review

BACKGROUND

In a period of change in the organization of primary care, Interprofessional Collaboration (IPC) is presented as one of the solutions to health issues. Although the number of inter-professional interventions grounded in primary care increases in all developed countries, evidence on the effects of these collaborations on patient-centred outcomes is patchy. The objective of our study was to assess the effects of IPC grounded in the primary care setting on patient-centred outcomes.

METHODS

We conducted a systematic literature review using the PubMed, Embase, PsycINFO and CINAHL databases from 01/01/1995 to 01/03/2021, according to the PRISMA guidelines. Studies reporting the effects of IPC in primary care on patient health outcomes were included. The quality of the studies was assessed using the revised Downs and Black checklist.

RESULTS

Sixty-five articles concerning 61 interventions were analysed. A total of 43 studies were prospective and randomized. Studies were classified into 3 main categories as follows: 1) studies with patients at cardiovascular risk (28 studies)-including diabetes (18 studies) and arterial hypertension (5 studies); 2) studies including elderly and/or polypathological patients (18 studies); and 3) patients with symptoms of mental or physical disorders (15 studies). The number of included patients varied greatly (from 50 to 312,377). The proportion of studies that reported a positive effect of IPC on patient-centred outcomes was as follows: 23 out of the 28 studies including patients at cardiovascular risk, 8 out of the 18 studies of elderly or polypathological patients, and 11 out of the 12 studies of patients with mental or physical disorders.

CONCLUSIONS

Evidence suggests that IPC is effective in the management of patients at cardiovascular risk. In elderly or polypathological patients and in patients with mental or physical disorders, the number of studies remains very limited, and the results are heterogeneous. Researchers should be encouraged to perform studies based on comparative designs: it would increase evidence on the positive effect and benefits of IPC on patient variables.

Biological sex disparity in survival outcomes following treatment for renal cell carcinoma: A systematic review and meta-analysis

INTRODUCTION

Renal cell carcinoma (RCC), a type of kidney cancer has biological sex-based differences that play a role in cancer incidence. Specifically, the incidence of urinary system cancers in men is two times greater than in women, while the incidence of genital cancers is three times greater. There is conflicting epidemiologic and limited evidence in the literature to suggest apparent biological sex discrepancy. The primary objective of this review and meta-analysis is to synthesize evidence to understand biological sex disparity in the survival outcomes of RCC following any treatment intervention.

METHODS

A three-step search strategy was utilized in this review. We searched MEDLINE, EMBASE and PsycINFO databases for manuscript on biological sex differences in treatment outcomes. Study screening, critical appraisal, and data extraction were executed independently by pairs of reviewers among co-authors. Studies that had any form of treatment modality in the management of RCC were included. Study designs included observational studies in the form of prospective and retrospective studies that utilized cox proportional hazard assumption to conduct survival analysis. The data synthesis was carried out using the R metafor software package (Software version of 1.2.8) and Microsoft Office Excel 2019 package (Microsoft Corporation, USA). The random effects model was estimated using restricted maximum likelihood estimation (REML). Data synthesis included narrative review and meta-analysis.

RESULTS

We had 23 eligible studies for this review. On review of the full text, 35 studies were excluded due to irrelevances to measure estimates utilized. Finally, 12 studies were selected for the meta-analysis with a total of n = 21,2453 individuals. Females had a better survival outcome following a treatment intervention for RCC than their male counterpart [Mean effect size = -0.1737 (95 % CI: -0.2524, -0.0949)].

CONCLUSION

Females were more likely to be cancer free than their male counterpart following treatment for RCC. This finding will inform appropriate decision making for stakeholders.

Increasing the efficiency of study selection for systematic reviews using prioritization tools and a single-screening approach

BACKGROUND

Systematic literature screening is a key component in systematic reviews. However, this approach is resource intensive as generally two persons independently of each other (double screening) screen a vast number of search results. To develop approaches for increasing efficiency, we tested the use of text mining to prioritize search results as well as the involvement of only one person (single screening) in the study selection process.

METHOD

Our study is based on health technology assessments (HTAs) of drug and non-drug interventions. Using a sample size calculation, we consecutively included 11 searches resulting in 33 study selection processes. Of the three screeners for each search, two used screening tools with prioritization (Rayyan, EPPI Reviewer) and one a tool without prioritization. For each prioritization tool, we investigated the proportion of citations classified as relevant at three cut-offs or STOP criteria (after screening 25%, 50% and 75% of the citation set). For each STOP criterion, we measured sensitivity (number of correctly identified relevant studies divided by the total number of relevant studies in the study pool). In addition, we determined the number of relevant studies identified per single screening round and investigated whether missed studies were relevant to the HTA conclusion.

RESULTS

Overall, EPPI Reviewer performed better than Rayyan and identified the vast majority (88%, Rayyan 66%) of relevant citations after screening half of the citation set. As long as additional information sources were screened, it was sufficient to apply a single-screening approach to identify all studies relevant to the HTA conclusion. Although many relevant publications (n = 63) and studies (n = 29) were incorrectly excluded, ultimately only 5 studies could not be identified at all in 2 of the 11 searches (1x 1 study, 1x 4 studies). However, their omission did not change the overall conclusion in any HTA.

CONCLUSIONS

EPPI Reviewer helped to identify relevant citations earlier in the screening process than Rayyan. Single screening would have been sufficient to identify all studies relevant to the HTA conclusion. However, this requires screening of further information sources. It also needs to be considered that the credibility of an HTA may be questioned if studies are missing, even if they are not relevant to the HTA conclusion.

The Past, Present, and Future of Psychotherapy Manuals: Protocol for a Scoping Review

BACKGROUND

Psychotherapy manuals are critical to the dissemination of psychotherapy treatments. Psychotherapy manuals typically serve several purposes, including, but not limited to, establishing new psychotherapy treatments, training providers, disseminating treatments to those who deliver them, and providing guidelines to deliver treatments with fidelity. Yet, the proliferation of psychotherapy manuals has not been well-understood, and no work has aimed to assess or review the existing landscape of psychotherapy manuals. Little is known about the breadth, scope, and foci of extant psychotherapy manuals.

OBJECTIVE

This scoping review aims to identify and explore the landscape of existing book-based psychotherapy manuals. This review aims to specify the defining characteristics (ie, foci, clinical populations, clinical targets, treatment type, treatment modality, and adaptations) of existing book-based psychotherapy manuals. Further, this review will demonstrate how this information, and psychotherapy manuals more broadly, has changed over time. This project aims to make a novel contribution that will have critical implications for current methods of developing, aggregating, synthesizing, and translating knowledge about psychotherapeutic treatments.

METHODS

This scoping review will review book-based psychotherapy manuals published from 1950 to 2022.This scoping review will be informed by guidance from the Joanna Briggs Institute Scoping Review Methodology Group and prior scoping reviews. Traditional search and application programming interface-based search methods will be used with search terms defined a priori to identify relevant results using 3 large book databases: Google Books, WorldCat, and PsycINFO. This review will leverage machine learning methods to enhance and expedite the screening process. Primary screening of results will be conducted by at least 2 authors. Data will be extracted and double-coded by research assistants using an iteratively defined codebook.

RESULTS

The search process produced 78,600 results, which were then iteratively deduplicated. Following deduplication, 50,583 results remained. The scoping review is expected to identify common elements of psychotherapy manuals, establish how the foci and content of manuals have changed over time, and illustrate coverage and gaps in the landscape of psychotherapy manuals. Results from this scoping review will be critical for future work focused on developing, aggregating, synthesizing, and disseminating knowledge about psychotherapeutic treatments.

CONCLUSIONS

This review will provide knowledge about the vast landscape of psychotherapy manuals that exist. Findings from this study will inform future efforts to develop, aggregate, synthesize, and translate knowledge about psychotherapeutic treatments.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/47708.

Performance of active learning models for screening prioritization in systematic reviews: a simulation study into the Average Time to Discover relevant records

BACKGROUND

Conducting a systematic review demands a significant amount of effort in screening titles and abstracts. To accelerate this process, various tools that utilize active learning have been proposed. These tools allow the reviewer to interact with machine learning software to identify relevant publications as early as possible. The goal of this study is to gain a comprehensive understanding of active learning models for reducing the workload in systematic reviews through a simulation study.

METHODS

The simulation study mimics the process of a human reviewer screening records while interacting with an active learning model. Different active learning models were compared based on four classification techniques (naive Bayes, logistic regression, support vector machines, and random forest) and two feature extraction strategies (TF-IDF and doc2vec). The performance of the models was compared for six systematic review datasets from different research areas. The evaluation of the models was based on the Work Saved over Sampling (WSS) and recall. Additionally, this study introduces two new statistics, Time to Discovery (TD) and Average Time to Discovery (ATD).

RESULTS

The models reduce the number of publications needed to screen by 91.7 to 63.9% while still finding 95% of all relevant records (WSS@95). Recall of the models was defined as the proportion of relevant records found after screening 10% of of all records and ranges from 53.6 to 99.8%. The ATD values range from 1.4% till 11.7%, which indicate the average proportion of labeling decisions the researcher needs to make to detect a relevant record. The ATD values display a similar ranking across the simulations as the recall and WSS values.

CONCLUSIONS

Active learning models for screening prioritization demonstrate significant potential for reducing the workload in systematic reviews. The Naive Bayes + TF-IDF model yielded the best results overall. The Average Time to Discovery (ATD) measures performance of active learning models throughout the entire screening process without the need for an arbitrary cut-off point. This makes the ATD a promising metric for comparing the performance of different models across different datasets.

The use of artificial intelligence for automating or semi-automating biomedical literature analyses: A scoping review

OBJECTIVE

Evidence-based medicine (EBM) is a decision-making process based on the conscious and judicious use of the best available scientific evidence. However, the exponential increase in the amount of information currently available likely exceeds the capacity of human-only analysis. In this context, artificial intelligence (AI) and its branches such as machine learning (ML) can be used to facilitate human efforts in analyzing the literature to foster EBM. The present scoping review aimed to examine the use of AI in the automation of biomedical literature survey and analysis with a view to establishing the state-of-the-art and identifying knowledge gaps.

MATERIALS AND METHODS

Comprehensive searches of the main databases were performed for articles published up to June 2022 and studies were selected according to inclusion and exclusion criteria. Data were extracted from the included articles and the findings categorized.

RESULTS

The total number of records retrieved from the databases was 12,145, of which 273 were included in the review. Classification of the studies according to the use of AI in evaluating the biomedical literature revealed three main application groups, namely assembly of scientific evidence (n = 127; 47%), mining the biomedical literature (n = 112; 41%) and quality analysis (n = 34; 12%). Most studies addressed the preparation of systematic reviews, while articles focusing on the development of guidelines and evidence synthesis were the least frequent. The biggest knowledge gap was identified within the quality analysis group, particularly regarding methods and tools that assess the strength of recommendation and consistency of evidence.

CONCLUSION

Our review shows that, despite significant progress in the automation of biomedical literature surveys and analyses in recent years, intense research is needed to fill knowledge gaps on more difficult aspects of ML, deep learning and natural language processing, and to consolidate the use of automation by end-users (biomedical researchers and healthcare professionals).

Financial strain and depression in the U.S.: a scoping review

While the association between assets and depression has been established, less is known about the link between financial strain and depression. Given rising financial strain and economic inequity due to the COVID-19 pandemic, understanding the role that financial strain plays in shaping population depression in the United States is particularly salient. We conducted a scoping review of the peer-reviewed literature on financial strain and depression published from inception through January 19, 2023, in Embase, Medline via PubMed, and PsycINFO, PsycArticles, SocINDEX, and EconLit via Ebsco. We searched, reviewed, and synthesized the literature on longitudinal studies on financial strain and depression conducted in the United States. Four thousand and four unique citations were screened for eligibility. Fifty-eight longitudinal, quantitative articles on adults in the United States were included in the review. Eighty-three percent of articles (n = 48) reported a significant, positive association between financial strain and depression. Eight articles reported mixed results, featuring non-significant associations for some sub-groups and significant associations for others, one article was unclear, and one article reported no significant association between financial strain and depression. Five articles featured interventions to reduce depressive symptoms. Effective interventions included coping mechanisms to improve one's financial situation (e.g., mechanisms to assist in finding employment), to modify cognitive behavior (e.g., reframing mindset), and to engage support (e.g., engaging social and community support). Successful interventions were tailored to participants, were group-based (e.g., they included family members or other job seekers), and occurred over multiple sessions. While depression was defined consistently, financial strain was defined variably. Gaps in the literature included studies featuring Asian populations in the United States and interventions to reduce financial strain. There is a consistent, positive association between financial strain and depression in the United States. More research is needed to identify and test interventions that mitigate the ill effects of financial strain on population's mental health.

Usefulness of machine learning softwares to screen titles of systematic reviews: a methodological study

OBJECTIVE

To investigate the usefulness and performance metrics of three freely-available softwares (Rayyan®, Abstrackr® and Colandr®) for title screening in systematic reviews.

STUDY DESIGN AND SETTING

In this methodological study, the usefulness of softwares to screen titles in systematic reviews was investigated by the comparison between the number of titles identified by software-assisted screening and those by manual screening using a previously published systematic review. To test the performance metrics, sensitivity, specificity, false negative rate, proportion missed, workload and timing savings were calculated. A purposely built survey was used to evaluate the rater's experiences regarding the softwares' performances.

RESULTS

Rayyan® was the most sensitive software and raters correctly identified 78% of the true positives. All three softwares were specific and raters correctly identified 99% of the true negatives. They also had similar values for precision, proportion missed, workload and timing savings. Rayyan®, Abstrackr® and Colandr® had 21%, 39% and 34% of false negatives rates, respectively. Rayyan presented the best performance (35/40) according to the raters.

CONCLUSION

Rayyan®, Abstrackr® and Colandr® are useful tools and provided good metric performance results for systematic title screening. Rayyan® appears to be the best ranked on the quantitative and on the raters' perspective evaluation. The most important finding of this study is that the use of software to screen titles does not remove any title that would meet the inclusion criteria for the final review, being valuable resources to facilitate the screening process.

Internal consistency and structural validity of the nomophobia questionnaire (NMP-Q) and its translations: A systematic review with meta-analysis

Background

A psychological condition known as NOMOPHOBIA causes individuals to fear losing ability to use or reach their mobile phones. The NMP-Q (nomophobia questionnaire) is a commonly used survey for assessing symptoms related to nomophobia.

Materials and methods

We performed a meta-analysis using reliability generalization (RG) on the NMP-Q. Thirteen studies involving 15,929 participants have reported original reliability estimates of the NMP-Q determined through a comprehensive and methodical examination of the available literature.

Results

For the total scores, the pooled internal consistency reliability was 0.93 [0.91; 0.95] and for the subscales it ranged from 0.83 to 0.91. Specifically, 0.91 [0.88; 0.93], 0.84 [0.80; 0.88], 0.83 [0.78; 0.88, and 0.83 [0.80; 0.85] for the subscales. Subscale 1 = not being able to communicate; subscale 2 = losing connectedness; subscale 3 = not being able to access information; and subscale 4 = giving up convenience", respectively. All reported effect sizes are Cronbach's alphas. Structural validity supported a solution of four-factors.

Conclusions

NMP-Q has an excellent internal consistency. Structural validity of four factors appears to be vigorous in fitting NMP-Q items. Our recommendation is to require future studies using NMP-Q to provide a reliability estimate based on their own data.

Characterizing Interoceptive Differences in Autism: A Systematic Review and Meta-analysis of Case-control Studies

Interoception, the body's perception of its own internal states, is thought to be altered in autism, though results of empirical studies have been inconsistent. The current study systematically reviewed and meta-analyzed the extant literature comparing interoceptive outcomes between autistic (AUT) and neurotypical (NT) individuals, determining which domains of interoception demonstrate robust between-group differences. A three-level Bayesian meta-analysis compared heartbeat counting performance, heartbeat discrimination performance, heartbeat counting confidence ratings, and self-reported interoceptive attention between AUT and NT groups (15 studies; nAUT = 467, nNT = 478). Autistic participants showed significantly reduced heartbeat counting performance [g = - 0.333, CrI95% (- 0.535, - 0.138)] and higher confidence in their heartbeat counting abilities [g = 0.430, CrI95% (0.123, 0.750)], but groups were equivalent on other meta-analyzed outcomes. Implications for future interoception research in autism are discussed.

Reducing Literature Screening Workload With Machine Learning

Abstract. In our era of accelerated accumulation of knowledge, the manual screening of literature for eligibility is increasingly becoming too labor-intensive for summarizing the current state of knowledge in a timely manner. Recent advances in machine learning and natural language processing promise to reduce the screening workload by automatically detecting unseen references with a high probability of inclusion. As a variety of tools have been developed, the current review provides an overview of their characteristics and performance. A systematic search in various databases yielded 488 eligible reports, revealing 15 tools for screening automation that differed in methodology, features, and accessibility. For the review on the performance of screening tools, 21 studies could be included. In comparison to sampling records randomly, active screening with prioritization approximately halves the screening workload. However, a comparison of tools under equal or at least similar conditions is needed to derive clear recommendations.

The Prevalence of Mild, Moderate, and Severe Nomophobia Symptoms: A Systematic Review, Meta-Analysis, and Meta-Regression

NOMOPHOBIA, or NO MObile PHone Phobia, refers to a psychological condition in which people fear being disconnected from their mobile phones. The purpose of this review was to establish the prevalence of nomophobia symptoms in youth and young adults according to severity, country, culture, population, measurement tool, and year of data collection. An electronic search of fourteen databases, two digital preservation services, and three content aggregator services was conducted from the inception of each database until 15 September 2021. A total of 52 studies involving 47,399 participants from 20 countries were included in the analyses. The prevalence of nomophobia was defined as the proportion of individuals scoring at or above established cut-offs on validated measures. Based on a random-effects meta-analysis, approximately 20% of individuals showed mild symptoms of nomophobia, 50% showed moderate symptoms, and 20% showed severe symptoms. Our results showed that university students from non-Western cultures are the most likely to suffer severe symptoms. In the year 2021, the prevalence rate of nomophobia increased. The instrument that was best able to detect nomophobia was the nomophobia questionnaire. Most individuals who own mobile phones experience mild or moderate symptoms of nomophobia. Severe symptoms deserve attention from clinicians and research scientists. A valid method of identifying individuals with a severe addiction to their mobile phones will help with timely and effective therapeutic management.

Increasing comprehensiveness and reducing workload in a systematic review of complex interventions using automated machine learning

BACKGROUND

As part of our ongoing systematic review of complex interventions for the primary prevention of cardiovascular diseases, we have developed and evaluated automated machine-learning classifiers for title and abstract screening. The aim was to develop a high-performing algorithm comparable to human screening.

METHODS

We followed a three-phase process to develop and test an automated machine learning-based classifier for screening potential studies on interventions for primary prevention of cardiovascular disease. We labelled a total of 16,611 articles during the first phase of the project. In the second phase, we used the labelled articles to develop a machine learning-based classifier. After that, we examined the performance of the classifiers in correctly labelling the papers. We evaluated the performance of the five deep-learning models [i.e. parallel convolutional neural network ( CNN ), stacked CNN , parallel-stacked CNN , recurrent neural network ( RNN ) and CNN-RNN]. The models were evaluated using recall, precision and work saved over sampling at no less than 95% recall.

RESULTS

We labelled a total of 16,611 articles, of which 676 (4.0%) were tagged as 'relevant' and 15,935 (96%) were tagged as 'irrelevant'. The recall ranged from 51.9% to 96.6%. The precision ranged from 64.6% to 99.1%. The work saved over sampling ranged from 8.9% to as high as 92.1%. The best-performing model was parallel CNN , yielding a 96.4% recall, as well as 99.1% precision, and a potential workload reduction of 89.9%.

FUTURE WORK AND LIMITATIONS

We used words from the title and the abstract only. More work needs to be done to look into possible changes in performance, such as adding features such as full document text. The approach might also not be able to be used for other complex systematic reviews on different topics.

CONCLUSION

Our study shows that machine learning has the potential to significantly aid the labour-intensive screening of abstracts in systematic reviews of complex interventions. Future research should concentrate on enhancing the classifier system and determining how it can be integrated into the systematic review workflow.

FUNDING

This project was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme and will be published in Health Technology Assessment. See the NIHR Journals Library website for further project information.

Economic evaluations of mammography to screen for breast cancer in low- and middle-income countries: A systematic review

Background

Low- and middle-income countries (LMICs) have limited resources compared to high-income countries (HICs). Therefore, it is critical that LMICs implement cost-effective strategies to reduce the burden of breast cancer. This study aimed to answer the question of whether mammography is a cost-effective breast cancer screening method in LMICs.

Methods

A systematic article search was conducted through Medline, Embase, Web of Science, and Econlit. Studies were included only if they conducted a full economic evaluation and focused on mammography screening in LMICs. Two reviewers screened through the title and abstract of each article and continued with full-text selection. Data were extracted and synthesized narratively. Quality assessment for each included study was conducted using the Consensus Health Economic Criteria (CHEC) extended checklist.

Results

This review identified 21 studies economically evaluating mammography as a breast cancer screening method in LMICs. Eighteen of these studies concluded that mammography screening was a cost-effective strategy. Most studies (71%) were conducted in upper-middle-income countries (Upper MICs). The quality of the studies varied from low to good. Important factors determining cost-effectiveness are the target age group (eg, 50-59 years), the screening interval (eg, biennial or triennial), as well as any combination with other breast cancer control strategies (eg, combination with treatment strategy for breast cancer patients).

Conclusions

Mammography screening appeared to be a cost-effective strategy in LMICs, particularly in Upper MICs. More studies conducted in lower-middle-income and low-income countries are needed to better understand the cost-effectiveness of mammography screening in these regions.

Emergent effects of global change on consumption depend on consumers and their resources in marine systems

Understanding the effects of global change on species interactions is important for predicting emergent ecosystem changes. Although environmental change can have direct effects on consumers, it is unclear if consumption will change in any generalizable way when both the consumer and resource(s) are exposed to future conditions. Using meta-analysis, we show high variability in consumption rates in response to ocean acidification and warming, indicating conclusions that suggest consumption will generally increase or decrease in a future ocean are premature. We also demonstrate how the interpretation is dependent on whether only the consumer or both the consumer and its resource(s) are exposed to future conditions. Based on these findings, we provide a road map for future research in this area.

A better understanding of how environmental change will affect species interactions would significantly aid efforts to scale up predictions of near-future responses to global change from individuals to ecosystems. To address this need, we used meta-analysis to quantify the individual and combined effects of ocean acidification (OA) and warming on consumption rates of predators and herbivores in marine ecosystems. Although the primary studies demonstrated that these environmental variables can have direct effects on consumers, our analyses highlight high variability in consumption rates in response to OA and warming. This variability likely reflects differences in local adaptation among species, as well as important methodological differences. For example, our results suggest that exposure of consumers to OA reduces consumption rates on average, yet consumption rates actually increase when both consumers and their resource(s) are concurrently exposed to the same conditions. We hypothesize that this disparity is due to increased vulnerability of prey or resource(s) in conditions of OA that offset declines in consumption. This hypothesis is supported by an analysis demonstrating clear declines in prey survival in studies that exposed only prey to future OA conditions. Our results illustrate how simultaneous OA and warming produce complex outcomes when species interact. Researchers should further explore other potential sources of variation in response, as well as the prey-driven component of any changes in consumption and the potential for interactive effects of OA and warming.

Using artificial intelligence methods for systematic review in health sciences: A systematic review

The exponential increase in published articles makes a thorough and expedient review of literature increasingly challenging. This review delineated automated tools and platforms that employ artificial intelligence (AI) approaches and evaluated the reported benefits and challenges in using such methods. A search was conducted in 4 databases (Medline, Embase, CDSR, and Epistemonikos) up to April 2021 for systematic reviews and other related reviews implementing AI methods. To be included, the review must use any form of AI method, including machine learning, deep learning, neural network, or any other applications used to enable the full or semi-autonomous performance of one or more stages in the development of evidence synthesis. Twelve reviews were included, using nine different tools to implement 15 different AI methods. Eleven methods were used in the screening stages of the review (73%). The rest were divided: two in data extraction (13%) and two in risk of bias assessment (13%). The ambiguous benefits of the data extractions, combined with the reported advantages from 10 reviews, indicating that AI platforms have taken hold with varying success in evidence synthesis. However, the results are qualified by the reliance on the self-reporting of the review authors. Extensive human validation still appears required at this stage in implementing AI methods, though further evaluation is required to define the overall contribution of such platforms in enhancing efficiency and quality in evidence synthesis.

Wealth and depression: A scoping review

INTRODUCTION

The inverse relation between income and depression is well established. Less is understood about the relation between wealth and depression. We therefore conducted a scoping review to answer the question: What is known from the existing literature about the relation between wealth and depression?

METHODS

We searched for studies and articles in Medline (via PubMed), Embase, PsycINFO, PsycArticles, EconLit, and SocINDEX from inception through July 19, 2020. Ninety-six articles were included in our review. Key article characteristics were year of publication, sample size, country, study design, definition of depression, definition of wealth, and association between wealth and depression. Thirty-two longitudinal articles were included in a detailed charted review.

RESULTS

Depression was defined in a relatively standard manner across articles. In contrast, definitions and measurements of wealth varied greatly. The majority of studies in the full review (n = 56, 58%) and half of the studies in the longitudinal charted review (n = 16, 50%) reported an inverse relation between wealth and depression. The longitudinal charted review showed that (1) macro-economic events influenced depression, (2) wealth status influenced depression across the lifecourse, (3) wealth protected against depression in the face of stressors such as job loss, (4) subjective or psychosocial factors such as perception of wealth, relative comparison, and social status modified the relation between wealth and depression, and (5) savings interventions were successful in reducing depression and varied by context.

CONCLUSION

These findings suggest that wealth should be included in our consideration of the forces that shape mental health.

Exclusive Maternal Milk Compared With Exclusive Formula on Growth and Health Outcomes in Very-Low-Birthweight Preterm Infants: Phase II of the Pre-B Project and an Evidence Analysis Center Systematic Review

As part of the Pre-B Project, a systematic review was conducted to evaluate associations between exclusive maternal milk (≥75%) intake and exclusive formula intake and growth and health outcomes in very-low-birthweight (VLBW) preterm infants. The protocols from the Academy of Nutrition and Dietetics' Evidence Analysis Center and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist were followed. Thirteen observational studies were included; 11 studies reported data that could be synthesized in a pooled analysis. The evidence is very uncertain (very low quality) about the effect of exclusive maternal milk on all outcomes due to observational study designs and risk of selection, performance, detection, and reporting bias in most of the included studies. Very-low-quality evidence suggested that providing VLBW preterm infants with exclusive maternal milk was not associated with mortality, risk of necrotizing enterocolitis, sepsis, or developing bronchopulmonary dysplasia, as compared with exclusive preterm formula, but exclusive maternal milk was associated with a lower risk of retinopathy of prematurity (very low certainty). Results may change when additional studies are conducted. There was no difference in weight, length, and head circumference gain between infants fed fortified exclusive maternal milk and infants receiving exclusive preterm formula; however, weight and length gain were lower in infants fed non-fortified exclusive maternal milk. Given the observational nature of human milk research, cause-and-effect evidence was lacking for VLBW preterm infants.

Systematic Review Registration

https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=86829, PROSPERO ID: CRD42018086829.

Effects of sediment exposure on corals: a systematic review of experimental studies

BACKGROUND

Management actions that address local-scale stressors on coral reefs can rapidly improve water quality and reef ecosystem condition. In response to reef managers who need actionable thresholds for coastal runoff and dredging, we conducted a systematic review and meta-analysis of experimental studies that explore the effects of sediment on corals. We identified exposure levels that 'adversely' affect corals while accounting for sediment bearing (deposited vs. suspended), coral life-history stage, and species, thus providing empirically based estimates of stressor thresholds on vulnerable coral reefs.

METHODS

We searched online databases and grey literature to obtain a list of potential studies, assess their eligibility, and critically appraise them for validity and risk of bias. Data were extracted from eligible studies and grouped by sediment bearing and coral response to identify thresholds in terms of the lowest exposure levels that induced an adverse physiological and/or lethal effect. Meta-regression estimated the dose-response relationship between exposure level and the magnitude of a coral's response, with random-effects structures to estimate the proportion of variance explained by factors such as study and coral species.

REVIEW FINDINGS

After critical appraisal of over 15,000 records, our systematic review of corals' responses to sediment identified 86 studies to be included in meta-analyses (45 studies for deposited sediment and 42 studies for suspended sediment). The lowest sediment exposure levels that caused adverse effects in corals were well below the levels previously described as 'normal' on reefs: for deposited sediment, adverse effects occurred as low as 1 mg/cm2/day for larvae (limited settlement rates) and 4.9 mg/cm2/day for adults (tissue mortality); for suspended sediment, adverse effects occurred as low as 10 mg/L for juveniles (reduced growth rates) and 3.2 mg/L for adults (bleaching and tissue mortality). Corals take at least 10 times longer to experience tissue mortality from exposure to suspended sediment than to comparable concentrations of deposited sediment, though physiological changes manifest 10 times faster in response to suspended sediment than to deposited sediment. Threshold estimates derived from continuous response variables (magnitude of adverse effect) largely matched the lowest-observed adverse-effect levels from a summary of studies, or otherwise helped us to identify research gaps that should be addressed to better quantify the dose-response relationship between sediment exposure and coral health.

CONCLUSIONS

We compiled a global dataset that spans three oceans, over 140 coral species, decades of research, and a range of field- and lab-based approaches. Our review and meta-analysis inform the no-observed and lowest-observed adverse-effect levels (NOAEL, LOAEL) that are used in management consultations by U.S. federal agencies. In the absence of more location- or species-specific data to inform decisions, our results provide the best available information to protect vulnerable reef-building corals from sediment stress. Based on gaps and limitations identified by our review, we make recommendations to improve future studies and recommend future synthesis to disentangle the potentially synergistic effects of multiple coral-reef stressors.

Perinatal diet and offspring anxiety: A scoping review

Health behaviors during pregnancy have an impact on the developing offspring. Dietary factors play a role in the development of mental illness: however, less is known about the impact of diet factors during pre-conception, gestation, and lactation on anxiety levels in offspring. This scoping review sought to systematically map the available research involving human and animal subjects to identify nutritional interventions which may have a harmful or protective effect, as well as identify gaps. Studies investigating an association between any perinatal diet pattern or diet constituent and offspring anxiety were included. The number of studies reporting an association with increased or decreased levels of anxiety were counted and presented in figures. A total of 55,914 results were identified as part of a larger scoping review, and 120 articles met the criteria for inclusion. A greater intake of phytochemicals and vitamins were associated with decreased offspring anxiety whereas maternal caloric restriction, protein restriction, reduced omega-3 consumption, and exposure to a high fat diet were associated with higher levels of offspring anxiety. Results were limited by a very large proportion of animal studies. High quality intervention studies involving human subjects are warranted to elucidate the precise dietary factors or constituents that modulate the risk of anxiety in offspring.

Food insecurity, type 2 diabetes, and hyperglycaemia: A systematic review and meta-analysis

AIMS

Food insecurity (FIS) is a major public health issue with possible implications for type 2 diabetes mellitus (T2DM) risk. We conducted a systematic review and meta-analysis to explore the association between FIS and T2DM.

METHODS

We performed a systematic search in PubMed, Embase, Scopus, and Web of Science. All cross-sectional, peer-reviewed studies investigating the link between FIS and T2DM were included. Population characteristics, study sizes, covariates, T2DM diagnoses, and diabetes-related clinical measures such as fasting blood glucose (FBG) and HbA1c were extracted from each study. Outcomes were compared between food insecure and food secure individuals. Effect sizes were combined across studies using the random effect model.

RESULTS

Forty-nine peer-reviewed studies investigating the link between FIS and T2DM were identified (n = 258,250). Results of meta-analyses showed no association between FIS and clinically determined T2DM either through FBG or HbA1c: OR = 1.22 [95%CI: 0.96, 1.55], Q(df = 5) = 12.5, I2  = 60% and OR = 1.21 [95%CI: 0.95, 1.54], Q(df = 5) = 14; I2  = 71% respectively. Standardized mean difference (SMD) meta-analyses yielded no association between FIS and FBG or HbA1c: g = 0.06 [95%CI: -0.06, 0.17], Q(df = 5) = 15.8, I2  = 68%; g = 0.11 [95% CI: -0.02, 0.25], Q(df = 7) = 26.8, I2  = 74% respectively. For children, no association was found between FIS and HbA1c: g = 0.06 [95%CI: 0.00, 0.17], Q(df = 2) = 5.7, I2  = 65%.

CONCLUSIONS

Despite multiple proposed mechanisms linking FIS to T2DM, integration of the available literature suggests FIS is not associated with clinically determined T2DM or increases in FBG or HbA1c among adult patients.

Guidance for using artificial intelligence for title and abstract screening while conducting knowledge syntheses

BACKGROUND

Systematic reviews are the cornerstone of evidence-based medicine. However, systematic reviews are time consuming and there is growing demand to produce evidence more quickly, while maintaining robust methods. In recent years, artificial intelligence and active-machine learning (AML) have been implemented into several SR software applications. As some of the barriers to adoption of new technologies are the challenges in set-up and how best to use these technologies, we have provided different situations and considerations for knowledge synthesis teams to consider when using artificial intelligence and AML for title and abstract screening.

METHODS

We retrospectively evaluated the implementation and performance of AML across a set of ten historically completed systematic reviews. Based upon the findings from this work and in consideration of the barriers we have encountered and navigated during the past 24 months in using these tools prospectively in our research, we discussed and developed a series of practical recommendations for research teams to consider in seeking to implement AML tools for citation screening into their workflow.

RESULTS

We developed a seven-step framework and provide guidance for when and how to integrate artificial intelligence and AML into the title and abstract screening process. Steps include: (1) Consulting with Knowledge user/Expert Panel; (2) Developing the search strategy; (3) Preparing your review team; (4) Preparing your database; (5) Building the initial training set; (6) Ongoing screening; and (7) Truncating screening. During Step 6 and/or 7, you may also choose to optimize your team, by shifting some members to other review stages (e.g., full-text screening, data extraction).

CONCLUSION

Artificial intelligence and, more specifically, AML are well-developed tools for title and abstract screening and can be integrated into the screening process in several ways. Regardless of the method chosen, transparent reporting of these methods is critical for future studies evaluating artificial intelligence and AML.

Diet and Anxiety: A Scoping Review

Anxiety disorders are the most common group of mental disorders. There is mounting evidence demonstrating the importance of nutrition in the development and progression of mental disorders such as depression; however, less is known about the role of nutrition in anxiety disorders. This scoping review sought to systematically map the existing literature on anxiety disorders and nutrition in order to identify associations between dietary factors and anxiety symptoms or disorder prevalence as well as identify gaps and opportunities for further research. The review followed established methodological approaches for scoping reviews. Due to the large volume of results, an online program (Abstrackr) with artificial intelligence features was used. Studies reporting an association between a dietary constituent and anxiety symptoms or disorders were counted and presented in figures. A total of 55,914 unique results were identified. After a full-text review, 1541 articles met criteria for inclusion. Analysis revealed an association between less anxiety and more fruits and vegetables, omega-3 fatty acids, "healthy" dietary patterns, caloric restriction, breakfast consumption, ketogenic diet, broad-spectrum micronutrient supplementation, zinc, magnesium and selenium, probiotics, and a range of phytochemicals. Analysis revealed an association between higher levels of anxiety and high-fat diet, inadequate tryptophan and dietary protein, high intake of sugar and refined carbohydrates, and "unhealthy" dietary patterns. Results are limited by a large percentage of animal and observational studies. Only 10% of intervention studies involved participants with anxiety disorders, limiting the applicability of the findings. High quality intervention studies involving participants with anxiety disorders are warranted.

Biological sex disparity in survival outcomes following treatment for renal cell carcinoma: a systematic review protocol

OBJECTIVE

This systematic review will assess the biological sex disparity in survival outcomes following treatment for renal cell carcinoma and analyze the estimates of biological sex disparity outcomes following supposed or proposed curative treatment.

INTRODUCTION

Renal cell carcinoma is a type of kidney cancer. There is a lack of conformity in the literature on the biological sex disparity in survival outcomes after treatment. This review will help inform the decision-making of clinicians, health care administrators, policy makers, public health workers, and pharmaceutical/biotechnology researchers in predicting positive outcomes following treatment.

INCLUSION CRITERIA

The review will consider prospective and retrospective studies on any form of treatment for renal cell carcinoma. The Cox proportional hazard assumption will be used to conduct survival analysis. Hazard rates of participants' survivability across biological sex will also be reported.

METHODS

A three-step search strategy will be used. First, a limited search of MEDLINE, Embase, and PsycINFO was conducted and text words in the title, abstract, and index terms were analyzed. Second, a search using identified keywords and index terms will be tailored for all included databases. Third, the reference lists of all included reports and articles will be screened to search for additional studies. There will be no language or date restrictions. Papers not written in English but with a professional translated copy will be included. Study screening, critical appraisal, and data extraction will be conducted independently by pairs of reviewers. Data synthesis will include narrative review and meta-analysis, if appropriate.

SYSTEMATIC REVIEW REGISTRATION NUMBER

PROSPERO CRD42020195721.

Transient Focal Neurological Events in Cerebral Amyloid Angiopathy and the Long-term Risk of Intracerebral Hemorrhage and Death: A Systematic Review and Meta-analysis

Importance

Transient focal neurological episodes (TFNEs) are a frequently overlooked presentation of cerebral amyloid angiopathy (CAA), a condition with prognostic implications that are still not well described.

Objective

To perform a systematic review and meta-analysis to examine the factors associated with incident lobar intracerebral hemorrhage (ICH) and death in patients with CAA presenting with TFNEs.

Data Sources

A systematic review and individual participant meta-analysis including (1) a hospital-based cohort and (2) the results obtained from a systematic search performed in MEDLINE and Embase completed in December 2019.

Study Selection

Included studies were observational reports of TFNEs. Patient-level clinical, imaging, and prognostic data were required for inclusion. For aggregate data studies, patient-level data were requested. Disagreements were resolved by consensus.

Data Extraction and Synthesis

Data were extracted following Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines by 4 independent reviewers. The quality of reports was determined based on the modified Pearson Case Report Quality Scale.

Main Outcomes and Measures

The clinical characteristics of TFNEs, neuroimaging features, and use of antithrombotics during follow-up were considered exposures. The predefined main outcomes were lobar ICH and risk of death during follow-up.

Results

Forty-two studies and 222 CAA-associated TFNE cases were included from the initial 1612 records produced by the systematic search; 26 additional patients (11 men [42.3%]; mean [SD] age, 77 [8] years) were provided by the hospital-based cohort. A total of 108 TFNEs (43.5%) consisted of motor symptoms. Convexity subarachnoid hemorrhage and cortical superficial siderosis were detected in 193 individuals (77.8%) and 156 individuals (62.9%) in the systematic search and hospital-based cohort, respectively. Follow-up duration could be obtained in 185 patients (median duration, 1 year [IQR, 0.8-2.5 years]). During follow-up, symptomatic lobar ICH occurred in 76 patients (39.4%). Motor symptoms (odds ratio, 2.08 [95% CI, 1.16-3.70]) at baseline and antithrombotic use during follow-up (odds ratio, 3.61 [95% CI, 1.67-7.84]) were associated with an increase in risk of lobar ICH. A total of 31 patients (16.5%) died during follow-up; lobar ICH during follow-up and cortical superficial siderosis were the main risk factors for death (odds ratio, 3.01 [95% CI, 1.36-6.69]; odds ratio, 3.20 [95% CI, 1.16-8.91], respectively).

Conclusions and Relevance

Patients presenting with CAA-associated TFNEs are at high risk of lobar ICH and death. Motor TFNEs and use of antithrombotics after a TFNE, in many cases because of misdiagnosis, are risk factors for ICH, and therefore accurate diagnosis and distinguishing this condition from transient ischemic attacks is critical.

The Impact of COVID-19 on the Majority Population, Ethno-Racial Minorities, and Immigrants

Abstract. The COVID-19 pandemic constitutes an unprecedented threat for individuals and societies, revealing stark inequalities in preparedness, exposure, and consequences. The present systematic literature review complements extant knowledge on disasters and pandemic diseases with programmatic research on the COVID-19 pandemic. Building upon an integrative definition of threat, we merge intra-personal threat regulation with group dynamics and inter-group relations. Via streamlined methods of knowledge synthesis, we first map out a broad taxonomy of threats, as appraised by the majority population and ethno-racial and immigrant minorities. Second, we delve into research linking threat appraisals with either conflict or prosociality within and across group boundaries. To conclude, we propose some guidelines for researchers to involve ethno-racial and immigrant minorities actively and for societies to cope cohesively with the impact of COVID-19.