Outbreaks of publications about emerging infectious diseases: the case of SARS-CoV-2 and Zika virus

Tool to assess risk of bias in studies estimating the prevalence of mental health disorders (RoB-PrevMH)

OBJECTIVE

There is no standard tool for assessing risk of bias (RoB) in prevalence studies. For the purposes of a living systematic review during the COVID-19 pandemic, we developed a tool to evaluate RoB in studies measuring the prevalence of mental health disorders (RoB-PrevMH) and tested inter-rater reliability.

METHODS

We decided on items and signalling questions to include in RoB-PrevMH through iterative discussions. We tested the reliability of assessments by different users with two sets of prevalence studies. The first set included a random sample of 50 studies from our living systematic review. The second set included 33 studies from a systematic review of the prevalence of post-traumatic stress disorders, major depression and generalised anxiety disorder. We assessed the inter-rater agreement by calculating the proportion of agreement and Kappa statistic for each item.

RESULTS

RoB-PrevMH consists of three items that address selection bias and information bias. Introductory and signalling questions guide the application of the tool to the review question. The inter-rater agreement for the three items was 83%, 90% and 93%. The weighted kappa scores were 0.63 (95% CI 0.54 to 0.73), 0.71 (95% CI 0.67 to 0.85) and 0.32 (95% CI -0.04 to 0.63), respectively.

CONCLUSIONS

RoB-PrevMH is a brief, user-friendly and adaptable tool for assessing RoB in studies on prevalence of mental health disorders. Initial results for inter-rater agreement were fair to substantial. The tool's validity, reliability and applicability should be assessed in future projects.

How to update a living systematic review and keep it alive during a pandemic: a practical guide

BACKGROUND

The covid-19 pandemic has highlighted the role of living systematic reviews. The speed of evidence generated during the covid-19 pandemic accentuated the challenges of managing high volumes of research literature.

METHODS

In this article, we summarise the characteristics of ongoing living systematic reviews on covid-19, and we follow a life cycle approach to describe key steps in a living systematic review.

RESULTS

We identified 97 living systematic reviews on covid-19, published up to 7th November 2022, which focused mostly on the effects of pharmacological interventions (n = 46, 47%) or the prevalence of associated conditions or risk factors (n = 30, 31%). The scopes of several reviews overlapped considerably. Most living systematic reviews included both observational and randomised study designs (n = 45, 46%). Only one-third of the reviews has been updated at least once (n = 34, 35%). We address practical aspects of living systematic reviews including how to judge whether to start a living systematic review, methods for study identification and selection, data extraction and evaluation, and give recommendations at each step, drawing from our own experience. We also discuss when it is time to stop and how to publish updates.

CONCLUSIONS

Methods to improve the efficiency of searching, study selection, and data extraction using machine learning technologies are being developed, their performance and applicability, particularly for reviews based on observational study designs should improve, and ways of publishing living systematic reviews and their updates will continue to evolve. Finally, knowing when to end a living systematic review is as important as knowing when to start.

A tool to assess risk of bias in studies estimating the prevalence of mental health disorders (RoB-PrevMH)

Objective

Biases affect how certain we are about the available evidence, however no standard tool for assessing the risk of bias (RoB) in prevalence studies exists. For the purposes of a living systematic review on prevalence of mental health disorders during the COVID-19 pandemic, we developed a RoB tool to evaluate prevalence studies in mental health (RoB-PrevMH) and tested interrater reliability.

Methods

We reviewed existing RoB tools for prevalence studies until September 2020, to develop a tool for prevalence studies in mental health. We tested the reliability of assessments by different users of RoB-PrevMH in 83 studies stemming from two systematic reviews of prevalence studies in mental health. We assessed the interrater agreement by calculating the proportion of agreement and Kappa statistic for each item.

Results

RoB-PrevMH consists of three items that address selection bias and information bias. Introductory and signaling questions guide the application of the tool to the review question. The interrater agreement for the three items was 83%, 90% and 93%. The weighted kappa was 0.63 (95% CI 0.54 to 0.73), 0.71 (95% CI 0.67 to 0.85) and 0.32 (95% CI -0.04 to -0.63), respectively.

Conclusions

We developed a brief, user friendly, and adaptable tool for assessing RoB in studies on prevalence of mental health disorders. Initial results for interrater agreement were fair to substantial. The tool's validity, reliability, and applicability should be assessed in future projects.

The Impact of the COVID-19 Pandemic and Associated Control Measures on the Mental Health of the General Population : A Systematic Review and Dose-Response Meta-analysis

BACKGROUND

To what extent the COVID-19 pandemic and its containment measures influenced mental health in the general population is still unclear.

PURPOSE

To assess the trajectory of mental health symptoms during the first year of the pandemic and examine dose-response relations with characteristics of the pandemic and its containment.

DATA SOURCES

Relevant articles were identified from the living evidence database of the COVID-19 Open Access Project, which indexes COVID-19-related publications from MEDLINE via PubMed, Embase via Ovid, and PsycInfo. Preprint publications were not considered.

STUDY SELECTION

Longitudinal studies that reported data on the general population's mental health using validated scales and that were published before 31 March 2021 were eligible.

DATA EXTRACTION

An international crowd of 109 trained reviewers screened references and extracted study characteristics, participant characteristics, and symptom scores at each timepoint. Data were also included for the following country-specific variables: days since the first case of SARS-CoV-2 infection, the stringency of governmental containment measures, and the cumulative numbers of cases and deaths.

DATA SYNTHESIS

In a total of 43 studies (331 628 participants), changes in symptoms of psychological distress, sleep disturbances, and mental well-being varied substantially across studies. On average, depression and anxiety symptoms worsened in the first 2 months of the pandemic (standardized mean difference at 60 days, -0.39 [95% credible interval, -0.76 to -0.03]); thereafter, the trajectories were heterogeneous. There was a linear association of worsening depression and anxiety with increasing numbers of reported cases of SARS-CoV-2 infection and increasing stringency in governmental measures. Gender, age, country, deprivation, inequalities, risk of bias, and study design did not modify these associations.

LIMITATIONS

The certainty of the evidence was low because of the high risk of bias in included studies and the large amount of heterogeneity. Stringency measures and surges in cases were strongly correlated and changed over time. The observed associations should not be interpreted as causal relationships.

CONCLUSION

Although an initial increase in average symptoms of depression and anxiety and an association between higher numbers of reported cases and more stringent measures were found, changes in mental health symptoms varied substantially across studies after the first 2 months of the pandemic. This suggests that different populations responded differently to the psychological stress generated by the pandemic and its containment measures.

PRIMARY FUNDING SOURCE

Swiss National Science Foundation. (PROSPERO: CRD42020180049).

Studies of prevalence: how a basic epidemiology concept has gained recognition in the COVID-19 pandemic

BACKGROUND

Prevalence measures the occurrence of any health condition, exposure or other factors related to health. The experience of COVID-19, a new disease caused by SARS-CoV-2, has highlighted the importance of prevalence studies, for which issues of reporting and methodology have traditionally been neglected.

OBJECTIVE

This communication highlights key issues about risks of bias in the design and conduct of prevalence studies and in reporting them, using examples about SARS-CoV-2 and COVID-19.

SUMMARY

The two main domains of bias in prevalence studies are those related to the study population (selection bias) and the condition or risk factor being assessed (information bias). Sources of selection bias should be considered both at the time of the invitation to take part in a study and when assessing who participates and provides valid data (respondents and non-respondents). Information bias appears when there are systematic errors affecting the accuracy and reproducibility of the measurement of the condition or risk factor. Types of information bias include misclassification, observer and recall bias. When reporting prevalence studies, clear descriptions of the target population, study population, study setting and context, and clear definitions of the condition or risk factor and its measurement are essential. Without clear reporting, the risks of bias cannot be assessed properly. Bias in the findings of prevalence studies can, however, impact decision-making and the spread of disease. The concepts discussed here can be applied to the assessment of prevalence for many other conditions.

CONCLUSIONS

Efforts to strengthen methodological research and improve assessment of the risk of bias and the quality of reporting of studies of prevalence in all fields of research should continue beyond this pandemic.

Occurrence and transmission potential of asymptomatic and presymptomatic SARS-CoV-2 infections: Update of a living systematic review and meta-analysis

BACKGROUND

Debate about the level of asymptomatic Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) infection continues. The amount of evidence is increasing and study designs have changed over time. We updated a living systematic review to address 3 questions: (1) Among people who become infected with SARS-CoV-2, what proportion does not experience symptoms at all during their infection? (2) What is the infectiousness of asymptomatic and presymptomatic, compared with symptomatic, SARS-CoV-2 infection? (3) What proportion of SARS-CoV-2 transmission in a population is accounted for by people who are asymptomatic or presymptomatic?

METHODS AND FINDINGS

The protocol was first published on 1 April 2020 and last updated on 18 June 2021. We searched PubMed, Embase, bioRxiv, and medRxiv, aggregated in a database of SARS-CoV-2 literature, most recently on 6 July 2021. Studies of people with PCR-diagnosed SARS-CoV-2, which documented symptom status at the beginning and end of follow-up, or mathematical modelling studies were included. Studies restricted to people already diagnosed, of single individuals or families, or without sufficient follow-up were excluded. One reviewer extracted data and a second verified the extraction, with disagreement resolved by discussion or a third reviewer. Risk of bias in empirical studies was assessed with a bespoke checklist and modelling studies with a published checklist. All data syntheses were done using random effects models. Review question (1): We included 130 studies. Heterogeneity was high so we did not estimate a mean proportion of asymptomatic infections overall (interquartile range (IQR) 14% to 50%, prediction interval 2% to 90%), or in 84 studies based on screening of defined populations (IQR 20% to 65%, prediction interval 4% to 94%). In 46 studies based on contact or outbreak investigations, the summary proportion asymptomatic was 19% (95% confidence interval (CI) 15% to 25%, prediction interval 2% to 70%). (2) The secondary attack rate in contacts of people with asymptomatic infection compared with symptomatic infection was 0.32 (95% CI 0.16 to 0.64, prediction interval 0.11 to 0.95, 8 studies). (3) In 13 modelling studies fit to data, the proportion of all SARS-CoV-2 transmission from presymptomatic individuals was higher than from asymptomatic individuals. Limitations of the evidence include high heterogeneity and high risks of selection and information bias in studies that were not designed to measure persistently asymptomatic infection, and limited information about variants of concern or in people who have been vaccinated.

CONCLUSIONS

Based on studies published up to July 2021, most SARS-CoV-2 infections were not persistently asymptomatic, and asymptomatic infections were less infectious than symptomatic infections. Summary estimates from meta-analysis may be misleading when variability between studies is extreme and prediction intervals should be presented. Future studies should determine the asymptomatic proportion of SARS-CoV-2 infections caused by variants of concern and in people with immunity following vaccination or previous infection. Without prospective longitudinal studies with methods that minimise selection and measurement biases, further updates with the study types included in this living systematic review are unlikely to be able to provide a reliable summary estimate of the proportion of asymptomatic infections caused by SARS-CoV-2.

REVIEW PROTOCOL

Open Science Framework (https://osf.io/9ewys/).