Cost-effectiveness of Microsoft Academic Graph with machine learning for automated study identification in a living map of coronavirus disease 2019 (COVID-19) research

Background

Identifying new, eligible studies for integration into living systematic reviews and maps usually relies on conventional Boolean updating searches of multiple databases and manual processing of the updated results. Automated searches of one, comprehensive, continuously updated source, with adjunctive machine learning, could enable more efficient searching, selection and prioritisation workflows for updating (living) reviews and maps, though research is needed to establish this. Microsoft Academic Graph (MAG) is a potentially comprehensive single source which also contains metadata that can be used in machine learning to help efficiently identify eligible studies. This study sought to establish whether: (a) MAG was a sufficiently sensitive single source to maintain our living map of COVID-19 research; and (b) eligible records could be identified with an acceptably high level of specificity.

Methods

We conducted an eight-arm cost-effectiveness analysis to assess the costs, recall and precision of semi-automated workflows, incorporating MAG with adjunctive machine learning, for continually updating our living map. Resource use data (time use) were collected from information specialists and other researchers involved in map production. Our systematic review software, EPPI-Reviewer, was adapted to incorporate MAG and associated machine learning workflows, and also used to collect data on recall, precision, and manual screening workload.

Results

The semi-automated MAG-enabled workflow dominated conventional workflows in both the base case and sensitivity analyses. At one month our MAG-enabled workflow with machine learning, active learning and fixed screening targets identified 469 additional, eligible articles for inclusion in our living map, and cost £3,179 GBP per week less, compared with conventional methods relying on Boolean searches of Medline and Embase.

Conclusions

We were able to increase recall and coverage of a large living map, whilst reducing its production costs. This finding is likely to be transferrable to OpenAlex, MAG's successor database platform.

Application and evolution of design in oral health: A systematic mapping study with an interactive evidence map

OBJECTIVES

There is increasing recognition of the value and capabilities of design in healthcare. Beyond the development of medical devices, design is increasingly being applied to intangible, complex and systemic healthcare problems. However, there is limited evidence on the use of design specifically in the field of oral health. This systematic mapping study aims to collate and catalogue evidence of design in oral health.

METHODS

A systematic search of academic databases and grey literature was performed. Duplicate results were removed, and publications relating to the same project were grouped. Reviewers from design and oral health independently screened a sample of the dataset. Projects of both relevance to oral health, and with input from a designer or clear implementation of a design methodology or approach were included. Projects were coded and plotted on a novel interactive evidence map.

RESULTS

119 design and oral health projects were included between 1973 and 2022. Interventional (n = 94, 79%), empirical (n = 46, 39%), methodological (n = 35, 29%) and theoretical (n = 7, 6%) design contributions were identified across the projects. The projects were categorized by four orders of design: first-graphics (n = 6, 5%), second-products (n = 41, 34%), third-interactions (n = 70, 59%), and fourth-systems (n = 2, 2%). Design was found in a diverse range of contexts in oral health; most commonly being relevant to general patients (n = 61, 51%), and for use in general dental practice (n = 56, 47%). Further design outcome categories (digital material; printed material; object; room or space; apparel; process; smart device; tangible interface; graphical interface; virtual reality; service; policy; system) and oral health themes (oral health literacy; oral care training; dental clinic design; dental instruments and equipment; personal oral care; dental appliance; clinician health and productivity; clinical information systems; informed consent; oral health promotion and prevention; oral care training; patient interactions and experience) were identified.

CONCLUSIONS

The novel interactive evidence map of design in oral health created enables ongoing and open-ended multivariant documentation and analysis of the evidence, as well as identification of strategic opportunities. Future research and policy implications include; recognition and engagement with the full capabilities of design; integration of design experts; fostering inclusive engagement and collaboration; disentangling patient and public involvement; advancing human-centred systems approaches; adopting design-led approaches for policy-making.

Interventions to support contraceptive choice and use: a global systematic map of systematic reviews

BACKGROUND

To review the highest level of available evidence, a systematic map identified systematic reviews that evaluated the effectiveness of interventions to improve contraception choice and increase contraception use.

METHODS

Systematic reviews published since 2000 were identified from searches of nine databases. Data were extracted using a coding tool developed for this systematic map. Methodological quality of included reviews was assessed using AMSTAR 2 criteria.

FINDINGS AND CONCLUSION

Fifty systematic reviews reported evaluations of interventions for contraception choice and use addressing three domains (individual, couples, community); Meta-analyses in 11 of the reviews mostly addressed interventions for individuals. We identified 26 reviews covering High Income Countries, 12 reviews covering Low Middle-Income Countries and the rest a mix of both. Most reviews (15) focussed on psychosocial interventions, followed by incentives (6) and m-health interventions (6). The strongest evidence from meta-analyses is for the effectiveness of motivational interviewing, contraceptive counselling, psychosocial interventions, school-based education, and interventions promoting contraceptive access, demand-generation interventions (community and facility based, financial mechanisms and mass media), and mobile phone message interventions. Even in resource constrained settings, community-based interventions can increase contraceptive use. There are gaps in the evidence on interventions for contraception choice and use, and limitations in study designs and lack of representativeness. Most approaches focus on individual women rather than couples or wider socio-cultural influences on contraception and fertility. This review identifies interventions which work to increase contraception choice and use, and these could be implemented in school, healthcare or community settings.

Animal models for cystic fibrosis: a systematic search and mapping review of the literature. Part 2: nongenetic models

Various animal models are available to study cystic fibrosis (CF). These models may help to enhance our understanding of the pathology and contribute to the development of new treatments. We systematically searched all publications on CF animal models. Because of the large number of models retrieved, we split this mapping review into two parts. Previously, we presented the genetic CF animal models. In this paper we present the nongenetic CF animal models. While genetic animal models may, in theory, be preferable for genetic diseases, the phenotype of a genetic model does not automatically resemble human disease. Depending on the research question, other animal models may thus be more informative.We searched Pubmed and Embase and identified 12,303 unique publications (after duplicate removal). All references were screened for inclusion by two independent reviewers. The genetic animal models for CF (from 636 publications) were previously described. The non-genetic CF models (from 189 publications) are described in this paper, grouped by model type: infection-based, pharmacological, administration of human materials, xenografts and other. As before for the genetic models, an overview of basic model characteristics and outcome measures is provided. This CF animal model overview can be the basis for an objective, evidence-based model choice for specific research questions. Besides, it can help to retrieve relevant background literature on outcome measures of interest.

Systematic map of conservation psychology

Conservation science and practice commonly draw on the theories and methods of social psychology to explain human cognition, emotion, and behavior germane to biodiversity conservation. We created a systematic map of the cross-disciplinary conservation science literature, which draws on social psychology concepts and methods in their application broadly described as conservation psychology. Established protocols were used to systematically collect and collate peer-reviewed research published in an explicit selection of multidisciplinary conservation journals. We sought to catalog the literature, elucidate trends and gaps, and critically reflect on the state of conservation psychology and its research practices that aim to influence conservation outcomes. The volume of publications per year and per decade increased from 1974 to 2016. Although a diversity of research designs and methods was applied, studies disproportionately focused on specific concepts (attitudes and beliefs), locations (North America and Europe), and contexts (terrestrial, rural). Studies also tended to be descriptive, quantitative, and atheoretical in nature. Our findings demonstrate that although conservation psychology has generally become more visible and prominent, it has done so within a limited space and suggest that disciplinary research principles and reporting standards must be more universally adopted by traditional and multidisciplinary conservation journals to raise the floor of empirical research.

Mapping the effectiveness of nature-based solutions for climate change adaptation

Nature-based solutions (NbS) to climate change currently have considerable political traction. However, national intentions to deploy NbS have yet to be fully translated into evidence-based targets and action on the ground. To enable NbS policy and practice to be better informed by science, we produced the first global systematic map of evidence on the effectiveness of nature-based interventions for addressing the impacts of climate change and hydrometeorological hazards on people. Most of the interventions in natural or semi-natural ecosystems were reported to have ameliorated adverse climate impacts. Conversely, interventions involving created ecosystems (e.g., afforestation) were associated with trade-offs; such studies primarily reported reduced soil erosion or increased vegetation cover but lower water availability, although this evidence was geographically restricted. Overall, studies reported more synergies than trade-offs between reduced climate impacts and broader ecological, social, and climate change mitigation outcomes. In addition, nature-based interventions were most often shown to be as effective or more so than alternative interventions for addressing climate impacts. However, there were substantial gaps in the evidence base. Notably, there were few studies of the cost-effectiveness of interventions compared to alternatives and few integrated assessments considering broader social and ecological outcomes. There was also a bias in evidence toward the Global North, despite communities in the Global South being generally more vulnerable to climate impacts. To build resilience to climate change worldwide, it is imperative that we protect and harness the benefits that nature can provide, which can only be done effectively if informed by a strengthened evidence base.

Development of a systematic mapping review protocol for the most recent evidence on ruminant infectious disease frequency and disease-associated mortality: Ethiopia as a case study

Almost 19% of the GDP of Ethiopia results from livestock production. Ruminants, in particular, form the majority of the national herd and are a critical source of income for smallholder farmers. Infectious diseases have been identified as a major cause of reduced livestock productivity in low- and middle-income countries (LMICs); therefore, a sound and comprehensive understanding of the relevant evidence would be beneficial in order to enable decision making on disease control policies. However, livestock disease data from sub-Saharan Africa is variable and disparate, which poses a challenge for evidence synthesis. This paper describes a protocol for a systematic mapping review of the recent available evidence on ruminant disease prevalence and associated mortality in Ethiopia. Literature sources will be identified using database search strategies. The titles, abstracts and, subsequently, full texts will be screened for inclusion based on predefined eligibility criteria. Specific data will be extracted and a preliminary qualitative assessment of the evidence will be performed using predefined indicators. The planned systematic map will be the first to provide a large-scale overview of the available ruminant disease evidence in Ethiopia; the final output will be an interactive dashboard tool to inform critical stakeholders in policy and research.

Cyberbullying and Children and Young People's Mental Health: A Systematic Map of Systematic Reviews

Cyberbullying is associated with considerable negative mental and psychosocial consequences in children and young people, making it a serious public health concern. To review the highest level of available evidence, a systematic mapping review was conducted to identify systematic reviews that investigated the relationship between cyberbullying and mental and psychological outcomes in young people. Topic-relevant bibliographic databases and online resources were searched to identify reviews published since 2007. Data were extracted using a coding tool developed for this study. Methodological quality of included reviews was assessed using AMSTAR criteria. Nineteen systematic reviews satisfied the inclusion criteria and they reported a strong negative association between cyberbullying and mental health outcomes in young people. Meta-analysis was performed in 11 reviews and narrative synthesis in 8 reviews. Data were derived from predominantly cross-sectional studies and a clear causal relationship between cyberbullying and mental outcomes cannot be established. Two-third of the included reviews were classified to be of low or unclear quality, due to the lack of quality assessment of the primary studies included in individual reviews. This systematic map consolidates available evidence at review level and confirms the existing gaps in longitudinal and qualitative evidence synthesis. Closer examination of the moderating factors influencing cyberbullying behaviors in future research can advance our understanding and inform the development of tailored programs of intervention to mitigate the negative impact of this phenomenon.

Animal models for cystic fibrosis: A systematic search and mapping review of the literature - Part 1: genetic models

Animal models for cystic fibrosis (CF) have enhanced our understanding of the pathology and contributed to the development of new treatments. In the field of CF, many animal models have been developed and described. To our knowledge, thus far, none of the reviews of CF animal models has used a systematic methodology. A systematic approach to creating model overviews can lead to an objective, evidence-based choice of an animal model for new research questions. We searched Pubmed and Embase for the currently available animal models for CF. Two independent reviewers screened the results. We included all primary studies describing an animal model for CF. After duplicate removal, 12,304 publications were left. Because of the large number of models, in the current paper, only the genetic models are presented. A total of 636 publications were identified describing genetic animal models for CF in mice, pigs, ferrets, rats and zebrafish. Most of these models have an altered Cftr gene. An overview of basic model characteristics and outcome measures for these genetic models is provided, together with advice on using these data. As far as the authors are aware, this is one of the largest systematic mapping reviews on genetic animal models for CF. It can aid in selecting a suitable model and outcome measures. In general, the reporting quality of the included publications was poor. Further systematic reviews are warranted to determine the quality and translational value of these models further.

Does sex really matter? Explaining intraspecies variation in ocean acidification responses

Ocean acidification (OA) poses a major threat to marine ecosystems globally, having significant ecological and economic importance. The number and complexity of experiments examining the effects of OA has substantially increased over the past decade, in an attempt to address multi-stressor interactions and long-term responses in an increasing range of aquatic organisms. However, differences in the response of males and females to elevated pCO₂ have been investigated in fewer than 4% of studies to date, often being precluded by the difficulty of determining sex non-destructively, particularly in early life stages. Here we highlight that sex can significantly impact organism responses to OA, differentially affecting physiology, reproduction, biochemistry and ultimately survival. What is more, these impacts do not always conform to ecological theory based on differential resource allocation towards reproduction, which would predict females to be more sensitive to OA owing to the higher production cost of eggs compared with sperm. Therefore, non-sex-specific studies may overlook subtle but ecologically significant differences in the responses of males and females to OA, with consequences for forecasting the fate of natural populations in a near-future ocean.

Making the Most of What We Already Know: A Three-Stage Approach to Systematic Reviewing

BACKGROUND

Conducting a systematic review in social policy is a resource-intensive process in terms of time and funds. It is thus important to understand the scope of the evidence base of a topic area prior to conducting a synthesis of primary research in order to maximize these resources. One approach to conserving resources is to map out the available evidence prior to undertaking a traditional synthesis. A few examples of this approach exist in the form of gap maps, overviews of reviews, and systematic maps supported by social policy and systematic review agencies alike. Despite this growing call for alternative approaches to systematic reviews, it is still common for systematic review teams to embark on a traditional in-depth review only.

OBJECTIVES

This article describes a three-stage approach to systematic reviewing that was applied to a systematic review focusing in interventions for smallholder farmers in Africa. We argue that this approach proved useful in helping us to understand the evidence base.

RESULTS

By applying preliminary steps as part of a three-stage approach, we were able to maximize the resources needed to conduct a traditional systematic review on a more focused research question. This enabled us to identify and fill real knowledge gaps, build on work that had already been done, and avoid wasting resources on areas of work that would have no useful outcome. It also facilitated meaningful engagement between the review team and our key policy stakeholders.

The data not collected on community forestry

Conservation and development practitioners increasingly promote community forestry as a way to conserve ecosystem services, consolidate resource rights, and reduce poverty. However, outcomes of community forestry have been mixed; many initiatives failed to achieve intended objectives. There is a rich literature on institutional arrangements of community forestry, but there has been little effort to examine the role of socioeconomic, market, and biophysical factors in shaping both land-cover change dynamics and individual and collective livelihood outcomes. We systematically reviewed the peer-reviewed literature on community forestry to examine and quantify existing knowledge gaps in the community-forestry literature relative to these factors. In examining 697 cases of community forest management (CFM), extracted from 267 peer-reviewed publications, we found 3 key trends that limit understanding of community forestry. First, we found substantial data gaps linking population dynamics, market forces, and biophysical characteristics to both environmental and livelihood outcomes. Second, most studies focused on environmental outcomes, and the majority of studies that assessed socioeconomic outcomes relied on qualitative data, making comparisons across cases difficult. Finally, there was a heavy bias toward studies on South Asian forests, indicating that the literature on community forestry may not be representative of decentralization policies and CFM globally.