Interpretation of Results of Studies Evaluating an Intervention Highlighted in Google Health News: A Cross-Sectional Study of News

Spin on adverse effects in abstracts of systematic reviews of orthodontic interventions: a cross-sectional study (part 2)

BACKGROUND

It is critical that abstracts of systematic reviews transparently report both the beneficial and adverse effects of interventions without misleading the readers. This cross-sectional study assessed whether adverse effects of interventions were reported or considered in abstracts of systematic reviews of orthodontic interventions and whether spin on adverse effects was identified when comparing the abstracts with what was sought and reported in these reviews.

METHODS

This cross-sectional study (part 2 of 2) used the same sample of 98 systematic reviews orthodontic interventions as used in part 1. Eligible reviews were retrieved from the Cochrane Database of Systematic Reviews and the 5 leading orthodontic journals between August 1 2009 and July 31 2021. Prevalence proportions were sought for 3 outcomes as defined in the published protocol. Univariable logistic regression models were built to explore associations between the presence of spin in the abstract and a series of predictors. Odds ratios (OR) 95% confidence intervals (95% CI) were used to quantify the strength of associations and their precision.

RESULTS

76.5% (75/98) of eligible reviews reported or considered (i.e., discussed, weighted etc.) potential adverse effects of orthodontic interventions in the abstract and the proportion of spin on adverse effects was 40.8% (40/98) in the abstract of these reviews. Misleading reporting was the predominant category of spin, i.e., 90% (36/40). Our explorative analyses found that compared to the Cochrane Database of Systematic Reviews all 5 orthodontic journals had similar odds of the presence of spin on adverse effects in abstracts of systematic reviews of orthodontic interventions. The odds of the presence of spin did not change over the sampled years (OR: 1.03, 95% CI: 0.9 to 1.16) and did not depend on the number of authors (OR: 0.93, 95% CI: 0.71 to 1.21), or on the type of orthodontic intervention (OR: 1.1, 95% CI: 0.45 to 2.67), or whether conflicts of interests were reported (OR: 0.74, 95% CI: 0.32 to 1.68).

CONCLUSION

End users of systematic reviews of orthodontic interventions have to be careful when interpreting results on adverse effects in the abstracts of these reviews, because they could be jeopardized by uncertainties such as not being reported and misleading reporting as a result of spin.

Systematic review finds "spin" practices and poor reporting standards in studies on machine learning-based prediction models

OBJECTIVES

We evaluated the presence and frequency of spin practices and poor reporting standards in studies that developed and/or validated clinical prediction models using supervised machine learning techniques.

STUDY DESIGN AND SETTING

We systematically searched PubMed from 01/2018 to 12/2019 to identify diagnostic and prognostic prediction model studies using supervised machine learning. No restrictions were placed on data source, outcome, or clinical specialty.

RESULTS

We included 152 studies: 38% reported diagnostic models and 62% prognostic models. When reported, discrimination was described without precision estimates in 53/71 abstracts (74.6% [95% CI 63.4-83.3]) and 53/81 main texts (65.4% [95% CI 54.6-74.9]). Of the 21 abstracts that recommended the model to be used in daily practice, 20 (95.2% [95% CI 77.3-99.8]) lacked any external validation of the developed models. Likewise, 74/133 (55.6% [95% CI 47.2-63.8]) studies made recommendations for clinical use in their main text without any external validation. Reporting guidelines were cited in 13/152 (8.6% [95% CI 5.1-14.1]) studies.

CONCLUSION

Spin practices and poor reporting standards are also present in studies on prediction models using machine learning techniques. A tailored framework for the identification of spin will enhance the sound reporting of prediction model studies.

A Descriptive Analysis of the Relationship between Social Media Use and Vaccine Hesitancy among a Sample of Unvaccinated Adults in Canada

Both traditional and social media information sources have disseminated information on the COVID-19 pandemic. The content shared may influence public opinion on different mitigation strategies, including vaccination. Misinformation can alter risk perception and increase vaccine hesitancy. This study aimed to explore the impact of using social media as the primary information source about the COVID-19 vaccine on COVID-19 vaccine hesitancy among people living in Canada. Secondary objectives identified other predictors of vaccine hesitancy and distinguished the effects of using traditional and social media sources. We used quota sampling of adults in Canada [N = 985] to conduct an online survey on the Pollfish survey platform between 21st and 28th May 2021. We then used bivariate chi-squared tests and multivariable logistic regression modeling to explore the associations between using social media as one's primary source of information about the COVID-19 vaccine and vaccine hesitancy. We further analyzed the association between specific types of channels of information and vaccine hesitancy. After controlling for covariates such as age, sex, race, and ethnicity, individuals reporting social media as their primary source of COVID-19 vaccine information versus those who had not used social media as their primary source of COVID-19 vaccine information had 50% higher odds of vaccine hesitancy. Among different channels of information, we found that information from television was associated with a 40% lower odds ratio for vaccine hesitancy. Since social media platforms play an essential role in influencing hesitancy in taking the COVID-19 vaccination, it is necessary to improve the quality of social media information sources and raise people's trust in information. Meanwhile, traditional media channels, such as television, are still crucial for promoting vaccination programs.

Resveratrol and Wine: An Overview of Thirty Years in the Digital News

BACKGROUND

Resveratrol's health benefits have received wide media coverage. Since resveratrol is usually associated with wine, informative texts about it should be prepared very carefully, since inaccurate website content could easily change people's wine consumption behavior. This study aimed to assess the quality of informative texts related to resveratrol on science journalism websites.

METHODS

We analyzed 125 resveratrol posts on Science Daily, WebMD, and EurekAlert! published between 1990 and 2020.

RESULTS

A higher number of posts was published in the years in which the number of people looking for information on the internet also increased. The increase can also be related to David Sinclair's notoriety, a fact that we called the "Sinclair effect". Most of the posts are replications of universities' press releases, mainly reporting resveratrol's health benefits, which resulted from preclinical studies and cannot be translated to humans. Most of them mention wine in the text and some in the title.

CONCLUSIONS

Wine is usually mentioned in headline resveratrol news, which could potentially influence wine consumption behavior. Scientists must intensify their efforts to communicate with the public to increase people's health literacy. Online news portals should have science journalists skilled in exploring scientific data and their translation into a simple and accurate language.

Causal and Associational Language in Observational Health Research: A Systematic Evaluation

We estimated the degree to which language used in the high-profile medical/public health/epidemiology literature implied causality using language linking exposures to outcomes and action recommendations; examined disconnects between language and recommendations; identified the most common linking phrases; and estimated how strongly linking phrases imply causality. We searched for and screened 1,170 articles from 18 high-profile journals (65 per journal) published from 2010-2019. Based on written framing and systematic guidance, 3 reviewers rated the degree of causality implied in abstracts and full text for exposure/outcome linking language and action recommendations. Reviewers rated the causal implication of exposure/outcome linking language as none (no causal implication) in 13.8%, weak in 34.2%, moderate in 33.2%, and strong in 18.7% of abstracts. The implied causality of action recommendations was higher than the implied causality of linking sentences for 44.5% or commensurate for 40.3% of articles. The most common linking word in abstracts was "associate" (45.7%). Reviewers' ratings of linking word roots were highly heterogeneous; over half of reviewers rated "association" as having at least some causal implication. This research undercuts the assumption that avoiding "causal" words leads to clarity of interpretation in medical research.

Spin in the Scientific Literature on Bariatric Endoscopy: a Systematic Review of Randomized Controlled Trials

Bariatric endoscopy (BE) is an emerging treatment option for people with obesity. Spin (i.e., the practice of frequent misrepresentation or overinterpretation of study findings) may lead to imbalanced and unjustified optimism in the interpretation of the results. The aim of this systematic review was to determine the frequency and type of spin in randomized controlled trials (RCTs) of endoscopic primary weight loss techniques with statistically significant and nonsignificant primary outcomes. In conclusion, spin is observed in the abstract and main text of BE reports and can lead to misinterpretation or overinterpretation of the results. Since BE challenges the available non-endoscopic treatments for obesity, further research is needed to better qualify these techniques, as being effective and safe, as well as predefined hypotheses and analyses.

They promised this ten years ago. Effects of diabetes news characteristics on patients' perceptions and attitudes towards medical innovations and therapy adherence

Patients have ever-increasing access to web-based news about hopeful scientific developments that may or may not cure them in the future. Science communication experts agree that the quality of news provision is not always guaranteed. However, literature does not clarify in what way users are actually affected by typical news characteristics such as the news object (described developmental phase of an innovation), the news source (degree of authority), and the news style (degree of language intensification). An online vignette experiment (N = 259) investigated causal relationships between characteristics of news about diabetes innovations and patients’ perceptions of future success, their interest in the innovation, and attitudes regarding current therapy adherence. Findings show that descriptions of success in mice led to higher estimations of future success chances than earlier and later developmental phases. Furthermore, news from a nonauthoritative source led to an increased interest in the innovation, and a more negative attitude towards current lifestyle advice. Lastly, the intensification of the language used in news messages showed slight adverse effects on the readers’ attitude. These findings, combined with their small effect sizes, support the optimistic view that diabetes patients are generally critical assessors of health news and that future research on this topic should focus on affected fragile subgroups.

Misrepresentation of mild traumatic brain injury research in press releases

BACKGROUND

Press releases from academic medical centers often form the basis for health and science news stories. Press release coverage of mild traumatic brain injury (mTBI) research has not been formally appraised in the literature.

OBJECTIVE

To perform a systematic content analysis of mTBI-based press releases.

DESIGN

Retrospective database study.

SETTING

EurekAlert! (eurekalert.org), the main distribution engine for scientific press releases.

PARTICIPANTS

Press releases indexed between January 1, 2012 and December 31, 2019 containing a minimum of 150 words.

INTERVENTIONS

Preestablished, investigator-generated criteria delineating aspects of misinformation.

MAIN OUTCOME MEASUREMENTS

Press releases were evaluated for manifestations of "spin," including misleading title, misleading reporting, misleading claims, and inappropriate extrapolation.

RESULTS

Our database search yielded 125 entries within the specified time period. Of these, 66 met inclusion criteria. Fifty-five of 66 (83%) press releases exhibited at least one manifestation of spin. We identified 38 (58%) with misleading titles, 49 (74%) with misleading reporting, 44 (67%) with misleading claims, and 38 (58%) with inappropriate extrapolation.

CONCLUSIONS

Our analysis revealed a high degree of spin in recent press releases dedicated to mTBI research. The reports often overstated the strengths and practical impact of the study, publicize substandard research without clinical relevance, while downplaying or failing to report limitations and caveats. Misrepresentation in press releases can affect real-life medical decisions and outcomes.

Quality of information in news media reports about the effects of health interventions: Systematic review and meta-analyses

Background Many studies have assessed the quality of news reports about the effects of health interventions, but there has been no systematic review of such studies or meta-analysis of their results. We aimed to fill this gap (PROSPERO ID: CRD42018095032). Methods We included studies that used at least one explicit, prespecified and generic criterion to assess the quality of news reports in print, broadcast, or online news media, and specified the sampling frame, and the selection criteria and technique. We assessed criteria individually for inclusion in the meta-analyses, excluding ineligible criteria and criteria with inadequately reported results. We mapped and grouped criteria to facilitate evidence synthesis. Where possible, we extracted the proportion of news reports meeting the included criterion. We performed meta-analyses using a random effects model to estimate such proportions for individual criteria and some criteria groups, and to characterise heterogeneity across studies.  Results We included 44 primary studies in the review, and 18 studies and 108 quality criteria in the meta-analyses. Many news reports gave an unbalanced and oversimplified picture of the potential consequences of interventions. A limited number mention or adequately address conflicts of interest (22%; 95% CI 7%-49%) (low certainty), alternative interventions (36%; 95% CI 26%-47%) (moderate certainty), potential harms (40%; 95% CI 23%-61%) (low certainty), or costs (18%; 95% CI 12%-28%) (moderate certainty), or quantify effects (53%; 95% CI 36%-69%) (low certainty) or report absolute effects (17%; 95% CI 4%-49%) (low certainty).  Discussion There is room for improving health news, but it is logically more important to improve the public's ability to critically appraise health information and make judgements for themselves.

Is rapid scientific publication also high quality? Bibliometric analysis of highly disseminated COVID-19 research papers

The impact of COVID-19 has underlined the need for reliable information to guide clinical practice and policy. This urgency has to be balanced against disruption to journal handling capacity and the continued need to ensure scientific rigour. We examined the reporting quality of highly disseminated COVID-19 research papers using a bibliometric analysis examining reporting quality and risk of bias (RoB) amongst 250 top scoring Altmetric Attention Score (AAS) COVID-19 research papers between January and April 2020. Method-specific RoB tools were used to assess quality. After exclusions, 84 studies from 44 journals were included. Forty-three (51%) were case series/studies, and only one was an randomized controlled trial. Most authors were from institutions based in China (n = 44, 52%). The median AAS and impact factor was 2015 (interquartile range [IQR] 1,105-4,051.5) and 12.8 (IQR 5-44.2) respectively. Nine studies (11%) utilized a formal reporting framework, 62 (74%) included a funding statement, and 41 (49%) were at high RoB. This review of the most widely disseminated COVID-19 studies highlights a preponderance of low-quality case series with few research papers adhering to good standards of reporting. It emphasizes the need for cautious interpretation of research and the increasingly vital responsibility that journals have in ensuring high-quality publications.

Quality of information in news media reports about the effects of health interventions: Systematic review and meta-analyses

Background Many studies have assessed the quality of news reports about the effects of health interventions, but there has been no systematic review of such studies or meta-analysis of their results. We aimed to fill this gap (PROSPERO ID: CRD42018095032). Methods We included studies that used at least one explicit, prespecified and generic criterion to assess the quality of news reports in print, broadcast, or online news media, and specified the sampling frame, and the selection criteria and technique. We assessed criteria individually for inclusion in the meta-analyses, excluding inappropriate criteria and criteria with inadequately reported results. We mapped and grouped criteria to facilitate evidence synthesis. Where possible, we extracted the proportion of news reports meeting the included criterion. We performed meta-analyses using a random effects model to estimate such proportions for individual criteria and some criteria groups, and to characterise heterogeneity across studies.  Results We included 44 primary studies in the qualitative summary, and 18 studies and 108 quality criteria in the meta-analyses. Many news reports gave an unbalanced and oversimplified picture of the potential consequences of interventions. A limited number mention or adequately address conflicts of interest (22%; 95% CI 7%-49%) (low certainty), alternative interventions (36%; 95% CI 26%-47%) (moderate certainty), potential harms (40%; 95% CI 23%-61%) (low certainty), or costs (18%; 95% CI 12%-28%) (moderate certainty), or quantify effects (53%; 95% CI 36%-69%) (low certainty) or report absolute effects (17%; 95% CI 4%-49%) (low certainty).  Discussion There is room for improving health news, but it is logically more important to improve the public's ability to critically appraise health information and make judgements for themselves.

Do not make clinical decisions based on abstracts of healthcare research: A systematic review

OBJECTIVE

To summarize the reporting quality of healthcare abstracts and inconsistencies between abstracts and full texts.

STUDY DESIGN AND SETTING

This systematic review included overviews of randomized controlled trials (RCTs) and systematic reviews (SRs) that summarized data of healthcare abstracts on reporting of abstracts and consistency of abstracts with the full text. Searches were performed in PubMed, CENTRAL, Cochrane Library and EMBASE databases from 1900 to February 2019. Two authors screened the overviews and extracted the data. All analyses were descriptive and divided into two main groups: abstracts' reporting quality and abstracts' consistency with the full text. Abstracts were considered poorly reported and inconsistent with the full text if more than 5% of abstracts' information was not fully reported or not consistent with the full text.

RESULTS

27 overviews analyzing 5,194 RCTs and 866 SRs were retrieved for reporting quality of abstracts. A total of 22 overviews analyzing 2,025 RCTs and 551 SRs were included for consistency of abstracts with the full text. Abstracts across all healthcare areas presented poor reporting quality and were inconsistent with the full texts, with results and conclusions as the most inconsistent sections.

CONCLUSION

Abstracts of healthcare RCTs and SRs have shown a large room for improvement in reporting quality and consistency with the full text. Authors, journal editors and reviewers need to give the highest priority to this matter.

How fast is the peer-review process for orthopaedic publications related to the Covid-19 pandemic?

PURPOSE

This study's primary objective is to evaluate the time spent in the peer-review process for orthopaedic publications related to the Covid-19 pandemic. The secondary objective is to evaluate the countries' and journals' contributions in these publications.

MATERIALS AND METHODS

We carried a search on August 1st, 2020, in one database (PubMed) using limited search terms to identify the orthopaedic publications related to the Covid-19 pandemic. After a filtration process, we evaluated the eligible article to identify the prevalence of different articles types, different countries, and journal contributions in these publications. The evaluation of the time spent in the peer-review process was done by obtaining the submission and acceptance dates.

RESULTS

Of the 231 articles eligible for initial assessment, review articles were the most common article type published (51%), 48 countries published articles in a unique 78 journals. Evaluation of the peer-review process in 147 articles revealed that the mean time from submission to acceptance was 14.3 ± 15.8 days (range from 0 to 74), the peer-review process took less than 30 days in 127 (86.4%) articles, 15 (10.2%) articles were accepted within the first day of submission.

CONCLUSION

Orthopaedic community contributed significantly to the publications related to the Covid-19 pandemic, with a contribution from many countries and journals. The peer-review process was notably shortened for some articles.

Evaluation of misrepresented findings in the abstracts of acute respiratory distress syndrome randomized trials with nonsignificant primary endpoints

OBJECTIVE

We investigated the randomized controlled trials (RCTs) related to acute respiratory distress syndrome (ARDS) to assess the presentation and frequency of misrepresented research findings, also known as spin.

METHODS

We searched PubMed (MEDLINE) for studies published from January 1, 2011 to December 31, 2019. We included randomized controlled trials with an ARDS intervention and a nonsignificant primary endpoint. Trial screening and data extraction was performed on all studies independently and in duplicate. The primary endpoint was to investigate the frequency and manifestation of spin in RCT abstracts. Our secondary endpoint was to investigate associations between funding source and spin.

RESULTS

Our PubMed search returned 766 articles with 37 meeting inclusion criteria. Spin was present in 14 (14/37, 37.8%; 95% CI 22.5%-55.2%) abstracts. The most common manifestations of spin were claiming benefit based on a significant secondary endpoint (6/14, 42.9%), followed by the use of 'trend' statements, such as 'trend toward significance' (2/14, 14.3%; 95% CI 1.8%-42.8%). The most common spin in abstract conclusions was in the form of claiming benefit due to a significant secondary endpoint (3/4, 75%; 95% CI 19.4%-99.4%). Our secondary endpoint did not identify a significant difference in the prevalence of spin in publicly funded (5/19, 26.3%; 95% CI 9.1%-51.2%) compared to privately funded (4/12, 33.3%; 95% CI 9.9%-65.1%) studies (p>.05).

CONCLUSIONS

RCTs of ARDS interventions with nonsignificant primary endpoints often included spin in the abstract. Spin in the abstract may influence clinician appraisal and interpretation of diagnostic or treatment modalities.

Clinical impact of high-profile animal-based research reported in the UK national press

OBJECTIVES

We evaluated animal-based biomedical 'breakthroughs' reported in the UK national press in 1995 (25 years prior to the conclusion of this study). Based on evidence of overspeculative reporting of biomedical research in other areas (eg, press releases and scientific papers), we specifically examined animal research in the media, asking, 'In a given year, what proportion of animal research "breakthroughs"' published in the UK national press had translated, more than 20 years later, to approved interventions?'

METHODS

We searched the Nexis media database (LexisNexis.com) for animal-based biomedical reports in the UK national press. The only restrictions were that the intervention should be specific, such as a named drug, gene, biomedical pathway, to facilitate follow-up, and that there should be claims of some clinical promise.

MAIN OUTCOME MEASURES

Were any interventions approved for human use? If so, when and by which agency? If not, why, and how far did development proceed? Were any other, directly related interventions approved? Did any of the reports overstate human relevance?

RESULTS

Overspeculation and exaggeration of human relevance was evident in all the articles examined. Of 27 unique published 'breakthroughs', only one had clearly resulted in human benefit. Twenty were classified as failures, three were inconclusive and three were partially successful.

CONCLUSIONS

The results of animal-based preclinical research studies are commonly overstated in media reports, to prematurely imply often-imminent 'breakthroughs' relevant to human medicine.

COVID-19: Pan(info)demic

The world has experienced pandemics worse than the coronavirus disease 2019 (COVID-19). However, the global effect of this pandemic has been overwhelming. Billions of people are in lockdown and also have easy access to information owing to easy and cheap internet connectivity and electronic media. However, information overload during the current COVID-19 pandemic has posed a set of challenges that were not encountered before. There is an ‘infodemic’ in which false news, conspiracy theories, magical cures, and racist news are being shared at an alarming rate, with the potential to increase anxiety and stress and even lead to loss of life. This review highlights some of these challenges and suggests general measures to avoid information overload and infodemic in the connected world of 21^st century.

Causal overstatements reduced in press releases following academic study of health news

Background: Exaggerations in health news were previously found to strongly associate with similar exaggerations in press releases. Moreover such exaggerations did not appear to attract more news. Here we assess whether press release practice changed after these reported findings; simply drawing attention to the issue may be insufficient for practical change, given the challenges of media environments. Methods: We assessed whether rates of causal over-statement in press releases based on correlational data were lower following a widely publicised paper on the topic, compared to an equivalent baseline period in the preceding year. Results: We found that over-statements in press releases were 28% (95% confidence interval = 16% to 45%) in 2014 and 13% (95% confidence interval = 6% to 25%) in 2015. A corresponding numerical reduction in exaggerations in news was not significant. The association between over-statements in news and press releases remained strong. Conclusions: Press release over-statements were less frequent following publication of Sumner et al. (2014). However, this is correlational evidence and the reduction may be due to other factors or natural fluctuations.

Effects of the Informed Health Choices podcast on the ability of parents of primary school children in Uganda to assess the trustworthiness of claims about treatment effects: one-year follow up of a randomised trial

INTRODUCTION

Earlier, we designed and evaluated an educational mass media intervention for improving people's ability to think more critically and to assess the trustworthiness of claims (assertions) about the benefits and harms (effects) of treatments. The overall aims of this follow-up study were to evaluate the impact of our intervention 1 year after it was administered, and to assess retention of learning and behaviour regarding claims about treatments.

METHODS

We randomly allocated consenting parents to listen to either the Informed Health Choices podcast (intervention) or typical public service announcements about health issues (control) over 7-10 weeks. Each intervention episode explained how the trustworthiness of treatment claims can be assessed by using relevant key concepts of evidence-informed decision-making. Participants listened to two episodes per week, delivered by research assistants. We evaluated outcomes immediately, and a year after the intervention. Primary outcomes were mean score and the proportion with a score indicating a basic ability to apply the key concepts (> 11 out of 18 correct answers) on a tool measuring people's ability to critically appraise the trustworthiness of treatment claims. Skills decay/retention was estimated by calculating the relative difference between the follow-up and initial results in the intervention group, adjusting for chance. Statistical analyses were performed using R (R Core Team, Vienna, Austria; version 3.4.3).

RESULTS

After 1 year, the mean score for parents in the intervention group was 58.9% correct answers, compared to 52.6% in the control (adjusted mean difference of 6.7% (95% CI 3.3% to 10.1%)). In the intervention group, 47.2% of 267 parents had a score indicating a basic ability to assess treatment claims compared to 39.5% of 256 parents in the control (adjusted difference of 9.8% more parents (95% CI 0.9% to 18.9%). These represent relative reductions of 29% in the mean scores and 33% in the proportion of parents with a score indicating a basic ability to assess the trustworthiness of claims about treatment effects.

CONCLUSIONS

Although listening to the Informed Health Choices podcast initially led to a large improvement in the ability of parents to assess claims about the effects of treatments, our findings show that these skills decreased substantially over 1 year. More active practice could address the substantial skills decay observed over 1 year.

TRIAL REGISTRATION

Pan African Clinical Trial Registry (www.pactr.org), PACTR201606001676150. Registered on 12 June 2016.

Causal claims about correlations reduced in press releases following academic study of health news

Background: Exaggerations in health news were previously found to strongly associate with similar exaggerations in press releases. Moreover such exaggerations did not appear to attract more news. Here we assess whether press release practice changed after these reported findings; simply drawing attention to the issue may be insufficient for practical change, given the challenges of media environments. Methods: We assessed whether rates of causal over-statement in press releases based on correlational data were lower following a seminal paper on the topic, compared to an equivalent baseline period in the preceding year. Results: We found that over-statements in press releases reduced from 28% (95% confidence interval = 16% to 45%) in 2014 to 13% (95% confidence interval = 6% to 25%) in 2015. A corresponding numerical reduction in exaggerations in news was not significant. The association between over-statements in news and press releases remained strong. Conclusions: Press release over-statements were less frequent following publication of Sumner et al. (2014), indicating that press release practice is malleable. However, this is correlational evidence and the reduction may be due to other factors.

Spin in the reporting, interpretation, and extrapolation of adverse effects of orthodontic interventions: protocol for a cross-sectional study of systematic reviews

Background

Titles and abstracts are the most read sections of biomedical papers. It is therefore important that abstracts transparently report both the beneficial and adverse effects of health care interventions and do not mislead the reader. Misleading reporting, interpretation, or extrapolation of study results is called “spin”. In this study, we will assess whether adverse effects of orthodontic interventions were reported or considered in the abstracts of both Cochrane and non-Cochrane reviews and whether spin was identified and what type of spin.

Methods

Eligibility criteria were defined for the type of study designs, participants, interventions, outcomes, and settings. We will include systematic reviews of clinical orthodontic interventions published in the five leading orthodontic journals and in the Cochrane Database. Empty reviews will be excluded. We will manually search eligible reviews published between 1 August 2009 and 31 July 2019. Data collection forms were developed a priori. All study selection and data extraction procedures will be conducted by two reviewers independently. Our main outcomes will be the prevalence of reported or considered adverse effects of orthodontic interventions in the abstract of systematic reviews and the prevalence of “spin” related to these adverse effects. We will also record the prevalence of three subtypes of spin, i.e., misleading reporting, misleading interpretation, and misleading extrapolation-related spin. All statistics will be calculated for the following groups: (1) all journals individually, (2) all journals together, and (3) the five leading orthodontic journals and the Cochrane Database of Systematic Reviews separately. Generalized linear models will be developed to compare the various groups.

Discussion

We expect that our results will raise the awareness of the importance of reporting and considering of adverse effects and the presence of the phenomenon of spin related to these effects in abstracts of systematic reviews of orthodontic interventions. This is important, because an incomplete and inadequate reporting, interpretation, or extrapolation of findings on adverse effects in abstracts of systematic reviews can mislead readers and could lead to inadequate clinical practice. Our findings could result in policy implications for making judgments about the acceptance for publication of systematic reviews of orthodontic interventions.

Digital aripiprazole or digital evergreening? A systematic review of the evidence and its dissemination in the scientific literature and in the media

BACKGROUND

In November 2017, the Food and Drug Administration (FDA) approved a version of a second-generation antipsychotic, aripiprazole, embedded with a sensor (Abilify MyCite).

OBJECTIVE

To systematically review the evidence supporting the FDA's approval of digital aripiprazole and how that evidence was disseminated in the scientific literature and news reports.

STUDY SELECTION

Prospective, double-blind, randomised controlled trials (RCTs), non-randomised and non-comparative studies were included if they focused on the use of digital aripiprazole. All scientific publications citing the trials were included if written in English. For the news reports, all languages were included if an English translation was available, and all records that were published after FDA approval were included.

FINDINGS

In the primary evidence search, no RCT comparing digital aripiprazole with a non-digital formulation, other active comparators or placebo was found. Only three non-comparative uncontrolled cohorts were found. No study provided data on remission, quality of life or any efficacy outcome. Fourteen scientific papers were identified that cited the trials and 70 news stories met the inclusion criteria. Almost 80% (11/14) of the scientific papers and three-fourths (52/70) of the news stories conveyed an unsupported impression of benefit.

CONCLUSIONS

Regulatory approval for this first-ever digital drug was based on weak evidence, and there was no evidence of better adherence with the digital version of aripiprazole compared with the non-digital version. The possibilities afforded by this technology make room for a new type of evergreening (ie, patenting of older drugs with a sensor as a 'new invention'). Both the scientific literature and news reports conveyed an unsupported impression of benefit.

TRIAL REGISTRATION NUMBER

CRD42018089515.

The association between exaggeration in health-related science news and academic press releases: a replication study

Background: Exaggerations in health news were previously found to strongly associate with similar exaggerations in press releases. Moreover, such press release exaggerations did not appear to attract more news.

Methods: Here we tested the replicability of these findings in a new cohort of news and press releases based on research in UK universities in 2014 and 2015. Press releases and news were compared to their associated peer-reviewed articles to define exaggeration in advice, causal claims and human inference from non-human studies.

Results: We found that the association between news and press releases did not replicate for advice exaggeration, while this association did replicate for causal claims and human inference from non-human studies. There was no evidence for higher news uptake for exaggerated press releases, consistent with previous results. Base exaggeration rates were lower for human inference from non-human studies, possibly reflecting the Concordat on Openness on Animal Research in the UK.

Conclusions: Overall, the picture remains that the strength of news statements is normally associated with the strength of press release statements, and without evidence that exaggerated statements get significantly more news.

The association between exaggeration in health-related science news and academic press releases: a replication study

Background: Exaggerations in health news were previously found to strongly associate with similar exaggerations in press releases. Moreover, such press release exaggerations did not appear to attract more news. Methods: Here we tested the replicability of these findings in a new cohort of news and press releases based on research in UK universities in 2014 and 2015. Press releases and news were compared to their associated peer-reviewed articles to define exaggeration in advice, causal claims and human inference from non-human studies. Results: We found that the association between news and press releases did not replicate for advice exaggeration, while this association did replicate for causal claims and human inference from non-human studies. There was no evidence for higher news uptake for exaggerated press releases, consistent with previous results. Base exaggeration rates were lower for human inference from non-human studies, possibly reflecting the Concordat on Openness on Animal Research in the UK. Conclusions: Overall, the picture remains that the strength of news statements is normally associated with the strength of press release statements, and without evidence that exaggerated statements get significantly more news.

Reporting gaps between news media and scientific papers on outdoor air pollution-related health outcomes: A content analysis

OBJECTIVES

In China, news media are useful for educating the public about the health threats of air pollution. To explore the potential gaps between scientific findings and the public's understanding of them, the characteristics of news media articles and their corresponding scientific papers were analysed.

METHODS

We used 22 articles relating to the health outcomes of exposure to outdoor air pollution published on Baidu News over the past year. An assessment tool developed by Robinson et al was used to evaluate the quality scores of news articles. Pearson correlation coefficients were used to measure the relationship between news media reporting and the characteristics of scientific papers. Misleading reporting, interpretation, or extrapolation in headlines and text bodies of news articles were examined.

RESULTS

The quality scores of the news articles ranged from -4 to 8, with an overall median score of 3. Correlation results showed that the scientific papers citation in Twitter (r = .88, P < .001) and Facebook (r = .64, P < .01) were significantly and positively associated with their citations in news stories. Media misunderstanding of scientific findings was common: 15 news headlines were identified with at least one spin (misrepresentation of scientific results), and 12 news articles had seven types of spin in the body texts.

CONCLUSION

Little media attention has been paid to scientific findings by Chinese researchers. Therefore, researchers and science journalists in China should make a better effort to engage in accurate and informative public discourse on domestic research.

Abstracts of low back pain trials are poorly reported, contain spin of information and are inconsistent with the full text: An overview study

OBJECTIVE

To investigate trials abstracts evaluating treatments for low back pain with regards to completeness of reporting, spin (i.e., interpretation of study results that overemphasizes the beneficial effects of the intervention), and inconsistencies in data with the full text.

DATA SOURCES

The search was performed on Physiotherapy Evidence Database (PEDro) in February 2016.

STUDY SELECTION

This is an overview study of a random sample of 200 low back pain trials published between 2010 and 2015. The languages of publication were restricted to English, Spanish and Portuguese.

DATA EXTRACTION

Completeness of reporting was assessed using the CONSORT for Abstracts checklist (CONSORT-A). Spin was assessed using a SPIN-checklist. Consistency between abstract and full text were assessed by applying the assessment tools to both the abstract and full text of each trial and calculating inconsistencies in the summary score (paired t test) and agreement in the classification of each item (Kappa statistics). Methodological quality was analyzed using the total PEDro score.

DATA SYNTHESIS

The mean number of fully reported items for abstracts using the CONSORT-A was 5.1 (SD 2.4) out of 15 points and the mean number of items with spin was 4.9 (SD 2.6) out of 7 points. Abstract and full text scores were statistically inconsistent (P=0.01). There was slight to moderate agreement between items of the CONSORT-A in the abstracts and full text (mean Kappa 0.20 SD 0.13) and fair to moderate agreement for items of the SPIN-checklist (mean Kappa 0.47 SD 0.09).

CONCLUSIONS

The abstracts were incomplete, with spin and inconsistent with the full text. We advise health care professionals to avoid making clinical decisions based solely upon abstracts. Journal editors, reviewers and authors are jointly responsible for improving abstracts, which could be guided by amended editorial policies.

Three randomized controlled trials evaluating the impact of "spin" in health news stories reporting studies of pharmacologic treatments on patients'/caregivers' interpretation of treatment benefit

BACKGROUND

News stories represent an important source of information. We aimed to evaluate the impact of "spin" (i.e., misrepresentation of study results) in health news stories reporting studies of pharmacologic treatments on patients'/caregivers' interpretation of treatment benefit.

METHODS

We conducted three two-arm, parallel-group, Internet-based randomized trials (RCTs) comparing the interpretation of news stories reported with or without spin. Each RCT considered news stories reporting a different type of study: (1) pre-clinical study, (2) phase I/II non-RCT, and (3) phase III/IV RCT. For each type of study, we identified news stories reported with spin that had earned mention in the press. Two versions of the news stories were used: the version with spin and a version rewritten without spin. Participants were patients/caregivers involved in Inspire, a large online community of more than one million patients/caregivers. The primary outcome was participants' interpretation assessed by one specific question "What do you think is the probability that 'treatment X' would be beneficial to patients?" (scale, 0 [very unlikely] to 10 [very likely]).

RESULTS

For each RCT, 300 participants were randomly assigned to assess a news story with spin (n = 150) or without spin (n = 150), and 900 participants assessed a news story. Participants were more likely to consider that the treatment would be beneficial to patients when the news story was reported with spin. The mean (SD) score for the primary outcome for abstracts reported with and without spin for pre-clinical studies was 7.5 (2.2) versus 5.8 (2.8) (mean difference [95% CI] 1.7 [1.0-2.3], p < 0.001); for phase I/II non-randomized trials, 7.6 (2.2) versus 5.8 (2.7) (mean difference 1.8 [1.0-2.5], p < 0.001); and for phase III/IV RCTs, 7.2 (2.3) versus 4.9 (2.8) (mean difference 2.3 [1.4-3.2], p < 0.001).

CONCLUSIONS

Spin in health news stories reporting studies of pharmacologic treatments affects patients'/caregivers' interpretation.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03094078 , NCT03094104 , NCT03095586.

Claims of causality in health news: a randomised trial

BACKGROUND

Misleading news claims can be detrimental to public health. We aimed to improve the alignment between causal claims and evidence, without losing news interest (counter to assumptions that news is not interested in communicating caution).

METHODS

We tested two interventions in press releases, which are the main sources for science and health news: (a) aligning the headlines and main causal claims with the underlying evidence (strong for experimental, cautious for correlational) and (b) inserting explicit statements/caveats about inferring causality. The 'participants' were press releases on health-related topics (N = 312; control = 89, claim alignment = 64, causality statement = 79, both = 80) from nine press offices (journals, universities, funders). Outcomes were news content (headlines, causal claims, caveats) in English-language international and national media (newspapers, websites, broadcast; N = 2257), news uptake (% press releases gaining news coverage) and feasibility (% press releases implementing cautious statements).

RESULTS

News headlines showed better alignment to evidence when press releases were aligned (intention-to-treat analysis (ITT) 56% vs 52%, OR = 1.2 to 1.9; as-treated analysis (AT) 60% vs 32%, OR = 1.3 to 4.4). News claims also followed press releases, significant only for AT (ITT 62% vs 60%, OR = 0.7 to 1.6; AT, 67% vs 39%, OR = 1.4 to 5.7). The same was true for causality statements/caveats (ITT 15% vs 10%, OR = 0.9 to 2.6; AT 20% vs 0%, OR 16 to 156). There was no evidence of lost news uptake for press releases with aligned headlines and claims (ITT 55% vs 55%, OR = 0.7 to 1.3, AT 58% vs 60%, OR = 0.7 to 1.7), or causality statements/caveats (ITT 53% vs 56%, OR = 0.8 to 1.0, AT 66% vs 52%, OR = 1.3 to 2.7). Feasibility was demonstrated by a spontaneous increase in cautious headlines, claims and caveats in press releases compared to the pre-trial period (OR = 1.01 to 2.6, 1.3 to 3.4, 1.1 to 26, respectively).

CONCLUSIONS

News claims-even headlines-can become better aligned with evidence. Cautious claims and explicit caveats about correlational findings may penetrate into news without harming news interest. Findings from AT analysis are correlational and may not imply cause, although here the linking mechanism between press releases and news is known. ITT analysis was insensitive due to spontaneous adoption of interventions across conditions.

TRIAL REGISTRATION

ISRCTN10492618 (20 August 2015).

Measuring semantic similarity of clinical trial outcomes using deep pre-trained language representations

BACKGROUND

Outcomes are variables monitored during a clinical trial to assess the impact of an intervention on humans' health.Automatic assessment of semantic similarity of trial outcomes is required for a number of tasks, such as detection of outcome switching (unjustified changes of pre-defined outcomes of a trial) and implementation of Core Outcome Sets (minimal sets of outcomes that should be reported in a particular medical domain).

OBJECTIVE

We aimed at building an algorithm for assessing semantic similarity of pairs of primary and reported outcomes.We focused on approaches that do not require manually curated domain-specific resources such as ontologies and thesauri.

METHODS

We tested several approaches, including single measures of similarity (based on strings, stems and lemmas, paths and distances in an ontology, and vector representations of phrases), classifiers using a combination of single measures as features, and a deep learning approach that consists in fine-tuning pre-trained deep language representations.We tested language models provided by BERT (trained on general-domain texts), BioBERT and SciBERT (trained on biomedical and scientific texts, respectively).We explored the possibility of improving the results by taking into account the variants for referring to an outcome (e.g.the use of a measurement tool name instead on the outcome name; the use of abbreviations).We release an open corpus with annotation for similarity of pairs of outcomes.

RESULTS

Classifiers using a combination of single measures as features outperformed the single measures, while deep learning algorithms using BioBERT and SciBERT models outperformed the classifiers.BioBERT reached the best F-measure of 89.75%.The addition of variants of outcomes did not improve the results for the best-performing single measures nor for the classifiers, but it improved the performance of deep learning algorithms: BioBERT achieved an F-measure of93.38%.

CONCLUSIONS

Deep learning approaches using pre-trained language representations outperformed other approaches for similarity assessment of trial outcomes, without relying on any manually curated domain-specific resources (ontologies and other lexical resources). Addition of variants of outcomes further improved the performance of deep learning algorithms.

Overinterpretation and misreporting of prognostic factor studies in oncology: a systematic review

BACKGROUND

Cancer prognostic biomarkers have shown disappointing clinical applicability. The objective of this study was to classify and estimate how study results are overinterpreted and misreported in prognostic factor studies in oncology.

METHODS

This systematic review focused on 17 oncology journals with an impact factor above 7. PubMed was searched for primary clinical studies published in 2015, evaluating prognostic factors. We developed a classification system, focusing on three domains: misleading reporting (selective, incomplete reporting, misreporting), misleading interpretation (unreliable statistical analysis, spin) and misleading extrapolation of the results (claiming irrelevant clinical applicability, ignoring uncertainty).

RESULTS

Our search identified 10,844 articles. The 98 studies included investigated a median of two prognostic factors (Q1-Q3, 1-7). The prognostic factors' effects were selectively and incompletely reported in 35/98 and 24/98 full texts, respectively. Twenty-nine articles used linguistic spin in the form of strong statements. Linguistic spin rejecting non-significant results was found in 34 full-text results and 15 abstract results sections. One in five articles had discussion and/or abstract conclusions that were inconsistent with the study findings. Sixteen reports had discrepancies between their full-text and abstract conclusions.

CONCLUSIONS

Our study provides evidence of frequent overinterpretation of findings of prognostic factor assessment in high-impact medical oncology journals.

Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review

Background

The pathway from evidence generation to consumption contains many steps which can lead to overstatement or misinformation. The proliferation of internet-based health news may encourage selection of media and academic research articles that overstate strength of causal inference. We investigated the state of causal inference in health research as it appears at the end of the pathway, at the point of social media consumption.

Methods

We screened the NewsWhip Insights database for the most shared media articles on Facebook and Twitter reporting about peer-reviewed academic studies associating an exposure with a health outcome in 2015, extracting the 50 most-shared academic articles and media articles covering them. We designed and utilized a review tool to systematically assess and summarize studies’ strength of causal inference, including generalizability, potential confounders, and methods used. These were then compared with the strength of causal language used to describe results in both academic and media articles. Two randomly assigned independent reviewers and one arbitrating reviewer from a pool of 21 reviewers assessed each article.

Results

We accepted the most shared 64 media articles pertaining to 50 academic articles for review, representing 68% of Facebook and 45% of Twitter shares in 2015. Thirty-four percent of academic studies and 48% of media articles used language that reviewers considered too strong for their strength of causal inference. Seventy percent of academic studies were considered low or very low strength of inference, with only 6% considered high or very high strength of causal inference. The most severe issues with academic studies’ causal inference were reported to be omitted confounding variables and generalizability. Fifty-eight percent of media articles were found to have inaccurately reported the question, results, intervention, or population of the academic study.

Conclusions

We find a large disparity between the strength of language as presented to the research consumer and the underlying strength of causal inference among the studies most widely shared on social media. However, because this sample was designed to be representative of the articles selected and shared on social media, it is unlikely to be representative of all academic and media work. More research is needed to determine how academic institutions, media organizations, and social network sharing patterns impact causal inference and language as received by the research consumer.

The Impact of a Case of Ebola Virus Disease on Emergency Department Visits in Metropolitan Dallas-Fort Worth, TX, July, 2013-July, 2015: An Interrupted Time Series Analysis

Background:

The first Ebola virus disease (EVD) case in the United States (US) was confirmed September 30, 2014 in a man 45 years old. This event created considerable media attention and there was fear of an EVD outbreak in the US.

Methods:

This study examined whether emergency department (ED) visits changed in metropolitan Dallas-Fort Worth­­, Texas (DFW) after this EVD case was confirmed. Using Texas Health Services Region 2/3 syndromic surveillance data and focusing on DFW, interrupted time series analyses were conducted using segmented regression models with autoregressive errors for overall ED visits and rates of several chief complaints, including fever with gastrointestinal distress (FGI). Date of fatal case confirmation was the “event.”

Results:

Results indicated the event was highly significant for ED visits overall (P<0.05) and for the rate of FGI visits (P<0.0001). An immediate increase in total ED visits of 1,023 visits per day (95% CI: 797.0, 1,252.8) was observed, equivalent to 11.8% (95% CI: 9.2%, 14.4%) increase ED visits overall. Visits and the rate of FGI visits in DFW increased significantly immediately after confirmation of the EVD case and remained elevated for several months even adjusting for seasonality both within symptom specific chief complaints as well as overall.

Conclusions:

These results have implications for ED surge capacity as well as for public health messaging in the wake of a public health emergency.

The U.S. Online News Coverage of Mammography Based on a Google News Search

RATIONALE AND OBJECTIVES

To characterize online news coverage relating to mammography, including articles' stance toward screening mammography.

MATERIALS AND METHODS

Google News was used to search U.S. news sites over a 9-year period (2006-2015) based on the search terms "mammography" and "mammogram." The top 100 search results were recorded. Identified articles were manually reviewed.

RESULTS

The top 100 news articles were from the following sources: local news outlet (50%), national news outlet (24%), nonimaging medical source (13%), entertainment or culture news outlet (6%), business news outlet (4%), peer-reviewed journal (1%), and radiology news outlet (1%). Most common major themes were the screening mammography controversy (29%), description of a new breast imaging technology (23%), dense breasts (11%), and promotion of a public screening initiative (11%). For the most recent year, article stance toward screening mammography was 59%, favorable; 16%, unfavorable; and 25%, neutral. After 2010, there was an abrupt shift in articles' stances from neutral to both favorable and unfavorable.

CONCLUSIONS

A wide range of online news sources addressed a range of issues related to mammography. National, rather than local, news sites were more likely to focus on the screening controversy and more likely to take an unfavorable view. The controversial United States Preventive Services Task Force guidelines may have influenced articles to take a stance on screening mammography. As such online news may impact public perception of the topic and thus potentially impact guideline adherence, radiologists are encouraged to maintain awareness of this online coverage and to support the online dissemination of reliable and accurate information.

Interpretation of health news items reported with or without spin: protocol for a prospective meta-analysis of 16 randomised controlled trials

INTRODUCTION

We aim to compare the interpretation of health news items reported with or without spin. 'Spin' is defined as a misrepresentation of study results, regardless of motive (intentionally or unintentionally) that overemphasises the beneficial effects of the intervention and overstates safety compared with that shown by the results.

METHODS AND ANALYSIS

We have planned a series of 16 randomised controlled trials (RCTs) to perform a prospective meta-analysis. We will select a sample of health news items reporting the results of four types of study designs, evaluating the effect of pharmacological treatment and containing the highest amount of spin in the headline and text. News items reporting four types of studies will be included: (1) preclinical studies; (2) phase I/II (non-randomised) trials; (3) RCTs and (4) observational studies. We will rewrite the selected news items and remove the spin. The original news and rewritten news will be appraised by four types of populations: (1) French-speaking patients; (2) French-speaking general public; (3) English-speaking patients and (4) English-speaking general public. Each RCT will explore the interpretation of news items reporting one of the four study designs by each type of population and will include a sample size of 300 participants. The primary outcome will be participants' interpretation of the benefit of treatment after reading the news items: (What do you think is the probability that treatment X would be beneficial to patients? (scale, 0 (very unlikely) to 10 (very likely)).This study will evaluate the impact of spin on the interpretation of health news reporting results of studies by patients and the general public.

ETHICS AND DISSEMINATION

This study has obtained ethics approval from the Institutional Review Board of the Institut national de la santé et de la recherche médicale (INSERM) (registration no: IRB00003888). The description of all the steps and the results of this prospective meta-analysis will be available online and will be disseminated as a published article. On the completion of this study, the results will be sent to all participants.

PROSPERO REGISTRATION NUMBER

CRD42017058941.

Establishing a library of resources to help people understand key concepts in assessing treatment claims-The "Critical thinking and Appraisal Resource Library" (CARL)

BACKGROUND

People are frequently confronted with untrustworthy claims about the effects of treatments. Uncritical acceptance of these claims can lead to poor, and sometimes dangerous, treatment decisions, and wasted time and money. Resources to help people learn to think critically about treatment claims are scarce, and they are widely scattered. Furthermore, very few learning-resources have been assessed to see if they improve knowledge and behavior.

OBJECTIVES

Our objectives were to develop the Critical thinking and Appraisal Resource Library (CARL). This library was to be in the form of a database containing learning resources for those who are responsible for encouraging critical thinking about treatment claims, and was to be made available online. We wished to include resources for groups we identified as 'intermediaries' of knowledge, i.e. teachers of schoolchildren, undergraduates and graduates, for example those teaching evidence-based medicine, or those communicating treatment claims to the public. In selecting resources, we wished to draw particular attention to those resources that had been formally evaluated, for example, by the creators of the resource or independent research groups.

METHODS

CARL was populated with learning-resources identified from a variety of sources-two previously developed but unmaintained inventories; systematic reviews of learning-interventions; online and database searches; and recommendations by members of the project group and its advisors. The learning-resources in CARL were organised by 'Key Concepts' needed to judge the trustworthiness of treatment claims, and were made available online by the James Lind Initiative in Testing Treatments interactive (TTi) English (www.testingtreatments.org/category/learning-resources).TTi English also incorporated the database of Key Concepts and the Claim Evaluation Tools developed through the Informed Healthcare Choices (IHC) project (informedhealthchoices.org).

RESULTS

We have created a database of resources called CARL, which currently contains over 500 open-access learning-resources in a variety of formats: text, audio, video, webpages, cartoons, and lesson materials. These are aimed primarily at 'Intermediaries', that is, 'teachers', 'communicators', 'advisors', 'researchers', as well as for independent 'learners'. The resources included in CARL are currently accessible at www.testingtreatments.org/category/learning-resources.

CONCLUSIONS

We hope that ready access to CARL will help to promote the critical thinking about treatment claims, needed to help improve healthcare choices.

Perspective: Improving Nutritional Guidelines for Sustainable Health Policies: Current Status and Perspectives

A large body of evidence supports the notion that incorrect or insufficient nutrition contributes to disease development. A pivotal goal is thus to understand what exactly is appropriate and what is inappropriate in food ingestion and the consequent nutritional status and health. The effective application of these concepts requires the translation of scientific information into practical approaches that have a tangible and measurable impact at both individual and population levels. The agenda for the future is expected to support available methodology in nutrition research to personalize guideline recommendations, properly grading the quality of the available evidence, promoting adherence to the well-established evidence hierarchy in nutrition, and enhancing strategies for appropriate vetting and transparent reporting that will solidify the recommendations for health promotion. The final goal is to build a constructive coalition among scientists, policy makers, and communication professionals for sustainable health and nutritional policies. Currently, a strong rationale and available data support a personalized dietary approach according to personal variables, including sex and age, circulating metabolic biomarkers, food quality and intake frequency, lifestyle variables such as physical activity, and environmental variables including one's microbiome profile. There is a strong and urgent need to develop a successful commitment among all the stakeholders to define novel and sustainable approaches toward the management of the health value of nutrition at individual and population levels. Moving forward requires adherence to well-established principles of evidence evaluation as well as identification of effective tools to obtain better quality evidence. Much remains to be done in the near future.

The development and validation of an instrument to measure the quality of health research reports in the lay media

Background

The media serves as an important link between medical research, as reported in scholarly sources, and the public and has the potential to act as a powerful tool to improve public health. However, concerns about the reliability of health research reports have been raised. Tools to monitor the quality of health research reporting in the media are needed to identify areas of weakness in health research reporting and to subsequently work towards the efficient use of the lay media as a public health tool through which the public’s health behaviors can be improved.

Methods

We developed the Quality Index for health-related Media Reports (QIMR) as a tool to monitor the quality of health research reports in the lay media. The tool was developed according to themes generated from interviews with health journalists and researchers. Item and domain characteristics and scale reliability were assessed. The scale was correlated with a global quality assessment score and media report word count to provide evidence towards its construct validity.

Results

The items and domains of the QIMR demonstrated acceptable validity and reliability. Items from the ‘validity’ domain were negatively skewed, suggesting possible floor effect. These items were not eliminated due to acceptable content and face validity. QIMR total scores produced a strong correlation with raters’ global assessment and a moderate correlation with media report word count, providing evidence towards the construct validity of the instrument.

Conclusions

The results of this investigation indicate that QIMR can adequately measure the quality of health research reports, with acceptable reliability and validity.

Electronic supplementary material

The online version of this article (doi:10.1186/s12889-017-4259-y) contains supplementary material, which is available to authorized users.