Reproducibility and replicability of systematic reviews

Climate-related health impact indicators for public health surveillance in a changing climate: a systematic review and local suitability analysis

Climate change challenges public health. Effective management of climate-related health risks relies on robust public health surveillance (PHS) and population health indicators. Despite existing global and country-specific indicators, their integration into local PHS systems is limited, impacting decision-making. We conducted a systematic review examining population health indicators relevant to climate change impacts and their suitability for national PHS systems. Guided by a registered protocol, we searched multiple databases and included 41 articles. Of these, 35 reported morbidity indicators, and 39 reported mortality indicators. Using Chile as a case study, we identified three sets of indicators for the Chilean PHS. The high-priority set included vector-, food-, and water-borne diseases, as well as temperature-related health outcomes indicators due to their easy integration into existing PHS systems. This review highlights the importance of population health indicators in monitoring climate-related health impacts, emphasising the need for local contextual factors to guide indicator selection.

Funding

This research project was partly funded by ANID Chile and University College London. None of these sources had any involvement in the research conceptualisation, design, or interpretation of the results.

Technology-based interventions for health challenges older women face amid COVID-19: a systematic review protocol

BACKGROUND

Pandemics, such as COVID-19, are dangerous and socially disruptive. Though no one is immune to COVID-19, older persons often bear the brunt of its consequences. This is particularly true for older women, as they often face more pronounced health challenges relative to other segments in society, including complex care needs, insufficient care provisions, mental illness, neglect, and increased domestic abuse. To further compound the situation, because protective measures like lockdowns can result in unintended consequences, many health services older women depend on can become disrupted or discontinued amid pandemics. While technology-based interventions have the potential to provide near-time, location-free, and virtually accessible care, there is a dearth of systematic insights into this mode of care in the literature. To bridge the research gaps, this investigation aims to examine the characteristics and effectiveness of technology-based interventions that could address health challenges older women face amid COVID-19.

METHODS

A systematic review of randomized trials reporting on technology-based interventions for older women (≥65 years) during COVID-19 will be conducted. The databases of Web of Science, ScienceDirect, PubMed/MEDLINE, PsycINFO, CINAHL, and Scopus will be searched. Retrieved citations will be screened independently by at least two reviewers against the eligibility criteria. Included studies will be assessed using the Cochrane ROB-2 tool. Data will be extracted independently by the reviewers. Where possible, meta-analyses will be performed on relevant study outcomes and analysed via odds ratios on the dichotomized outcomes. Where applicable, heterogeneity will be measured using the Cochrane Q test, and publication bias will be assessed via funnel plots and Egger's regression test.

DISCUSSION

Technology has the potential to transform healthcare for the better. To help society better safeguard vulnerable populations' health and quality of life, this investigation sets out to gauge the state-of-the-art development of technology-based interventions tailored to the health challenges older women face amid COVID-19. In light of the growing prevalence of population ageing and the inevitability of infectious disease outbreaks, greater research efforts are needed to ensure the timely inception and effective implementation of technology-based health solutions for vulnerable populations like older women, amid public health crises like COVID-19 and beyond.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42020194003.

A new method for testing reproducibility in systematic reviews was developed, but needs more testing

BACKGROUND

To develop and test an approach to test reproducibility of SRs.

METHODS

Case study. We have developed an approach to test reproducibility retrospectively while focusing on the whole conduct of an SR instead of single steps of it. We replicated the literature searches and drew a 25% random sample followed by study selection, data extraction, and risk of bias (ROB) assessments performed by two reviewers independently. These results were compared narratively with the original review.

RESULTS

We were not able to fully reproduce the original search resulting in minor differences in the number of citations retrieved. The biggest disagreements were found in study selection. The most difficult section to be reproduced was the RoB assessment due to the lack of reporting clear criteria to support the judgement of RoB ratings, although agreement was still found to be satisfactory.

CONCLUSION

Our approach as well as other approaches needs to undergo testing and comparison in the future as the area of testing for reproducibility of SRs is still in its infancy.

Structured reviews for data and knowledge-driven research

Hypothesis generation is a critical step in research and a cornerstone in the rare disease field. Research is most efficient when those hypotheses are based on the entirety of knowledge known to date. Systematic review articles are commonly used in biomedicine to summarize existing knowledge and contextualize experimental data. But the information contained within review articles is typically only expressed as free-text, which is difficult to use computationally. Researchers struggle to navigate, collect and remix prior knowledge as it is scattered in several silos without seamless integration and access. This lack of a structured information framework hinders research by both experimental and computational scientists. To better organize knowledge and data, we built a structured review article that is specifically focused on NGLY1 Deficiency, an ultra-rare genetic disease first reported in 2012. We represented this structured review as a knowledge graph and then stored this knowledge graph in a Neo4j database to simplify dissemination, querying and visualization of the network. Relative to free-text, this structured review better promotes the principles of findability, accessibility, interoperability and reusability (FAIR). In collaboration with domain experts in NGLY1 Deficiency, we demonstrate how this resource can improve the efficiency and comprehensiveness of hypothesis generation. We also developed a read–write interface that allows domain experts to contribute FAIR structured knowledge to this community resource. In contrast to traditional free-text review articles, this structured review exists as a living knowledge graph that is curated by humans and accessible to computational analyses. Finally, we have generalized this workflow into modular and repurposable components that can be applied to other domain areas. This NGLY1 Deficiency-focused network is publicly available at http://ngly1graph.org/.

Availability and implementation

Database URL: http://ngly1graph.org/. Network data files are at: https://github.com/SuLab/ngly1-graph and source code at: https://github.com/SuLab/bioknowledge-reviewer.

Contact

asu@scripps.edu

Study-based registers reduce waste in systematic reviewing: discussion and case report

BACKGROUND

Maintained study-based registers (SBRs) have, at their core, study records linked to, potentially, multiple other records such as references, data sets, standard texts and full-text reports. Such registers can minimise and refine searching, de-duplicating, screening and acquisition of full texts. SBRs can facilitate new review titles/updates and, within seconds, inform the team about the potential workload of each task.

METHODS

We discuss the advantages/disadvantages of SBRs and report a case of how such a register was used to develop a successful grant application and deliver results-reducing considerable redundancy of effort.

RESULTS

SBRs saved time in question-setting and scoping and made rapid production of nine Cochrane systematic reviews possible.

CONCLUSION

Whilst helping prioritise and conduct systematic reviews, SBRs improve quality. Those funding information specialists for literature reviewing could reasonably stipulate the resulting SBR to be delivered for dissemination and use beyond the life of the project.