Is thematic analysis used well in health psychology? A critical review of published research, with recommendations for quality practice and reporting

Despite the persistent dominance of a 'scientific psychology' paradigm in health psychology, the use of qualitative research continues to grow. Qualitative approaches are often based on fundamentally different values from (post)positivistempiricism, raising important considerations for quality, and whether qualitative work adheres to, and is judged by, appropriate publication standards. Thematic analysis (TA) has become a particularly popular method in qualitative health psychology, but poor practice is widespread. To support high quality, methodologically coherent TA practice and reporting, we critically reviewed 100 systematically selected papers reporting TA, published in five prominent health psychology journals. Our review assessed reported practice, and considered this in relation to methodological and quality recommendations. We identified 10 common areas of problematic practice in the reviewed papers, the majority citing reflexive TA. Considering the role of three 'arbiters of quality' in a peer review publication system - authors, reviewers, and editors - we developed 20 recommendations for authors, to support them in conducting and reporting high quality TA research, with associated questions for reviewers and editors to consider when evaluating TA manuscripts for publication. We end with considerations for facilitating better qualitative research, and enriching the understandings and knowledge base from which health psychology is practiced.

Around the EQUATOR with Clin-STAR: Prediction modeling opportunities and challenges in aging research

The 2015 Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Statement was published to improve reporting transparency for prediction modeling studies. The objective of this review is to highlight methodologic challenges that aging-focused researchers will encounter when designing and reporting studies involving prediction models for older adults and provide guidance for addressing these challenges. In following the 22-item TRIPOD checklist, researchers must consider the representativeness of cohorts used (e.g., whether older adults with frailty, cognitive impairment, and social isolation were included), strategies for incorporating common geriatric predictors (e.g., age, comorbidities, functional status, and frailty), methods for handling missing data and competing risk of death, and assessment of model performance heterogeneity across important subgroups (e.g., age, sex, race, and ethnicity). We provide guidance to help aging-focused researchers develop, validate, and report models that can inform and improve patient care, which we label "TRIPOD-65."

Standardization of In Vitro Studies for Plasmonic Photothermal therapy

Lack of standardization is a systematic problem that impacts nanomedicine by challenging data comparison from different studies. Translation from preclinical to clinical stages indeed requires reproducible data that can be easily accessed and compared. In this work, we propose a series of experimental standards for in vitro plasmonic photothermal therapy (PPTT). This best practice guide covers the five main aspects of PPTT studies in vitro: nanomaterials, biological samples, pre-, during, and postirradiation characterization. We are confident that such standardization of experimental protocols and reported data will benefit the development of PPTT as a transversal therapy.

Disclosure Standards for Social Media and Generative Artificial Intelligence Research: Toward Transparency and Replicability

Social media dominate today's information ecosystem and provide valuable information for social research. Market researchers, social scientists, policymakers, government entities, public health researchers, and practitioners recognize the potential for social data to inspire innovation, support products and services, characterize public opinion, and guide decisions. The appeal of mining these rich datasets is clear. However, there is potential risk of data misuse, underscoring an equally huge and fundamental flaw in the research: there are no procedural standards and little transparency. Transparency across the processes of collecting and analyzing social media data is often limited due to proprietary algorithms. Spurious findings and biases introduced by artificial intelligence (AI) demonstrate the challenges this lack of transparency poses for research. Social media research remains a virtual "wild west," with no clear standards for reporting regarding data retrieval, preprocessing steps, analytic methods, or interpretation. Use of emerging generative AI technologies to augment social media analytics can undermine validity and replicability of findings, potentially turning this research into a "black box" enterprise. Clear guidance for social media analyses and reporting is needed to assure the quality of the resulting research. In this article, we propose criteria for evaluating the quality of studies using social media data, grounded in established scientific practice. We offer clear documentation guidelines to ensure that social data are used properly and transparently in research and applications. A checklist of disclosure elements to meet minimal reporting standards is proposed. These criteria will make it possible for scholars and practitioners to assess the quality, credibility, and comparability of research findings using digital data.

Measurement of radiologist reporting times: Assessment of precision, comparison of three different measurement techniques and review of potential applications

INTRODUCTION

Radiologist reporting times are a key component of radiology department workload assessment, but reliable measurement remains challenging. Currently, there are three contenders for this task: median reporting times (MRTs), extracted directly from a department's radiology information system (RIS); study-ascribed times (SATs), using published tables of individual descriptors derived from a combination of measurement and consensus; and radiology reporting figures (RRFs), using published tables of measured times based on modality and numbers of anatomical areas.

METHODS

We review these techniques, their possible uses and some potential pitfalls. We discuss the level of precision that can realistically be attained in measuring reporting times, and list the strengths and weaknesses of each technique, comparing them in relation to each of eight potential applications.

RESULTS

We believe that SATs are challenging for practical use due to their static nature, absent common descriptors and large number. RRFs are more user-friendly but are also static and require ongoing updates; currently, they do not include ultrasound. MRTs cannot currently be extracted from every RIS, but where available they are easy to use and their dynamic nature provides the most objective data. They underestimate the unmeasurable components of a radiologist's work and therefore the total time spent in a reporting session.

CONCLUSION

MRTs are superior to the other methods in flexibility, precision and ease of use. All institutions should have access to this data and we call on vendors of Radiology Information Systems which are currently not capable of providing it to make the necessary modifications.

Systematic Review of Descriptions and Justifications Provided for Single-Case Quantification Techniques

There are currently a multitude of quantification techniques that have been developed for use with single-case designs. As a result, choosing an appropriate quantification technique can be overwhelming and it can be difficult for researchers to properly describe and justify their use of quantification techniques. However, providing clear descriptions and justifications is important for enhancing the credibility of single-case research, and allowing others to evaluate the appropriateness of the quantification technique used. The aim of this systematic literature review is to provide an overview of the quantification techniques that are used to analyze single-case designs, with a focus on the descriptions and justifications that are provided. A total of 290 quantifications occurred across 218 articles, and the descriptions and justifications that were provided for the quantification techniques that were used are systematically examined. Results show that certain quantification techniques, such as the non-overlap indices, are more commonly used. Descriptions and justifications provided for using the quantification techniques are sometimes vague or subjective. Single-case researchers are encouraged to complement visual analysis with the use of quantification techniques for which they can provide objective and appropriate descriptions and justifications, and are encouraged to use tools to guide their choice of quantification techniques.

iCHECK-DH: Guidelines and Checklist for the Reporting on Digital Health Implementations

BACKGROUND

Implementation of digital health technologies has grown rapidly, but many remain limited to pilot studies due to challenges, such as a lack of evidence or barriers to implementation. Overcoming these challenges requires learning from previous implementations and systematically documenting implementation processes to better understand the real-world impact of a technology and identify effective strategies for future implementation.

OBJECTIVE

A group of global experts, facilitated by the Geneva Digital Health Hub, developed the Guidelines and Checklist for the Reporting on Digital Health Implementations (iCHECK-DH, pronounced "I checked") to improve the completeness of reporting on digital health implementations.

METHODS

A guideline development group was convened to define key considerations and criteria for reporting on digital health implementations. To ensure the practicality and effectiveness of the checklist, it was pilot-tested by applying it to several real-world digital health implementations, and adjustments were made based on the feedback received. The guiding principle for the development of iCHECK-DH was to identify the minimum set of information needed to comprehensively define a digital health implementation, to support the identification of key factors for success and failure, and to enable others to replicate it in different settings.

RESULTS

The result was a 20-item checklist with detailed explanations and examples in this paper. The authors anticipate that widespread adoption will standardize the quality of reporting and, indirectly, improve implementation standards and best practices.

CONCLUSIONS

Guidelines for reporting on digital health implementations are important to ensure the accuracy, completeness, and consistency of reported information. This allows for meaningful comparison and evaluation of results, transparency, and accountability and informs stakeholder decision-making. i-CHECK-DH facilitates standardization of the way information is collected and reported, improving systematic documentation and knowledge transfer that can lead to the development of more effective digital health interventions and better health outcomes.

Adverse Effects of Virtual and Augmented Reality Interventions in Psychiatry: Systematic Review

BACKGROUND

Virtual reality (VR) and augmented reality (AR) are emerging treatment modalities in psychiatry, which are capable of producing clinical outcomes broadly comparable to those achieved with standard psychotherapies.

OBJECTIVE

Because the side effect profile associated with the clinical use of VR and AR remains largely unknown, we systematically reviewed available evidence of their adverse effects.

METHODS

A systematic review was conducted in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) framework across 3 mental health databases (PubMed, PsycINFO, and Embase) to identify VR and AR interventions targeting mental health diagnoses.

RESULTS

Of 73 studies meeting the inclusion criteria, 7 reported worsening clinical symptoms or an increased fall risk. Another 21 studies reported "no adverse effects" but failed to identify obvious adverse effects, mainly cybersickness, documented in their results. More concerningly, 45 of the 73 studies made no mention of adverse effects whatsoever.

CONCLUSIONS

An appropriate screening tool would help ensure that VR adverse effects are correctly identified and reported.

Around the EQUATOR with clinician-scientists transdisciplinary aging research (Clin-STAR) principles: Implementation science challenges and opportunities

The Institute of Medicine and the National Institute on Aging increasingly understand that knowledge alone is necessary but insufficient to improve healthcare outcomes. Adapting the behaviors of clinicians, patients, and stakeholders to new standards of evidence-based clinical practice is often significantly delayed. In response, over the past twenty years, Implementation Science has developed as the study of methods and strategies that facilitate the uptake of evidence-based practice into regular use by practitioners and policymakers. One important advance in Implementation Science research was the development of Standards for Reporting Implementation Studies (StaRI), which provided a 27-item checklist for researchers to consistently report essential elements of the implementation and intervention strategies. Using StaRI as a framework, this review discusses specific Implementation Science challenges for research with older adults, provides solutions for those obstacles, and opportunities to improve the value of this evolving approach to reduce the knowledge translation losses that exist between published research and clinical practice.

Transparent transdisciplinary reporting in geriatric research using the EQUATOR Network

This editorial comments on the article by Carpenter et al. in this issue.

[Application of "patient and public involvement" in acupuncture clinical research]

In order to explore the application of "patient and public involvement" (PPI) in acupuncture clinical research, the connotation, reporting standards and research status of PPI at home and abroad are collated, and the key problems of PPI encountered in acupuncture clinical research are deeply considered and summarized. It is suggested that the short-form checklist of the Guidance for Reporting Involvement of Patients and the Public (GRIPP) of the 2nd edition should be applied to acupuncture clinical research. PPI provides a new perspective for acupuncture clinical research. It is beneficial for each stage of research, contributes to the improvement of acupuncture medical service mode and increases the success rate and cost-effectiveness of research so that the innovation and development of acupuncture science can be promoted.

Whole exome sequencing studies in epilepsy: A deep analysis of the published literature

To evaluate the quality of whole-exome sequencing (WES) reporting in the epilepsy literature. We aimed to assess the quality of reporting of WES in epilepsy. We compared studies based on journal type and if outcome reporting biases exist. We used a self-constructed benchmark to quantitatively analyze studies. We included 451 publications. Reporting was heterogeneous with poor reporting of (1) ACMG guideline application 13% and (2) Human Phenotype Ontology (HPO) numbers in 3% of studies, 3) VUS in 19%. Predictors of reporting included journal type and journal impact factor. Date of publication and publication type were not predictors of poor reporting. Pairwise comparisons of genetics versus neurology journals using relative risks yielded significant differences in reporting of ACMG guideline application (RR 1.88, 95% CI 1.04-3.38); HPO numbers (RR 8.62, 95% CI 1.08-63.37) and deposition of findings to ClinVar (RR 2.50, 95% CI 1.03-6.1). Reporting of WES literature is heterogeneous in quality, and poor reporting hinders collaboration and accession of data into large databases like OMIM and OrphaNet. This study highlights reporting bias in this area and, formal structural guidelines like the CONSORT guidelines used in the reporting of clinical trials are needed to address the issue.

Estándares Consolidados de Reporte de Evaluaciones Económicas Sanitarias: adaptación al español de la lista de comprobación CHEERS 2022

OBJECTIVES

Health economic evaluations (HEEs) are comparative analyses of courses of action in terms of both costs and consequences. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) original version and its adaptation to Spanish were published in 2013. Its objectives were to promote that the HEEs are identifiable, interpretable, and useful for decision making and serve as a reporting guide. The new CHEERS 2022 replaces the previous one and tries to be more easily applied to any HEE and incorporates recent methodological advances and the importance of stakeholder involvement including patients and the general public.

METHODS

For the present adaptation, the following stages were followed: (1) independent translations of the original list into Spanish, (2) blind back-translations, (3) evaluation of their quality, (4) preparation of a new version in Spanish, (5) review and improvement by the author team, (6) preparation of a new version in Spanish, (7) distribution of the preliminary Spanish version and the original one to the American HTA Network (Red de las Américas de Evaluación de Tecnologías Sanitarias) and Spanish-speaking experts for evaluation and feedback, (8) monitoring of changes to the original list under peer review at BritishMedicalJournal, and (9) consolidation of the final adaptation of the Spanish CHEERS 2022 checklist.

RESULTS

In this article, we detail the process and the Spanish adaptation of the 28-item CHEERS 2022 checklist and its recommendations.

CONCLUSIONS

This list is intended for researchers reporting HEE in peer-reviewed journals and reviewers, editors, and, among others, health technology assessment bodies.

What have we learned?

As the final outputs of the Reproducibility Project: Cancer Biology are published, it is clear that preclinical research in cancer biology is not as reproducible as it should be.

Misidentified biomedical resources: Journal guidelines are not a quick fix

Biomedical researchers routinely use a variety of biological models and resources, such as cultured cell lines, antibodies and laboratory animals. Unfortunately, these resources are not flawless: cell lines can be misidentified; for antibodies, problems with specificity, lot‐to‐lot consistency and sensitivity are common; and the reliability of animal models is questioned due to poor translation of animal studies to human clinical trials. In some cases, these problems can render the results of a study meaningless. As a response, some journals have implemented guidelines regarding the use and reporting of cell lines, antibodies and laboratory animals. In our study we use a portfolio of existing and newly created datasets to investigate identification and authentication information of cell lines, antibodies and organisms before and after guideline introduction, compared to journals without guidelines. We observed a general improvement of reporting quality over time, which the implementation of guidelines accelerated only in some cases. We therefore conclude that the effectiveness of journal guidelines is likely to be context dependent, affected by factors such as implementation conditions, research community support and monitoring and resource availability. Hence, journal reporting guidelines in themselves are not a quick fix to repair shortcomings in biomedical resource documentation, even though they can be part of the solution.

Longitudinal validity of self-rated health: the presence and impact of response shift

Objective: This paper aimed to examine the longitudinal validity of self-rated health (SRH) and whether it would be affected by possible changes in evaluation standards (i.e., response shift) over time.Design: Data are from a longitudinal survey of a nationally representative sample in China. Analytical sample was restricted to respondents aged 45 and above (n = 15,893). Individual fixed effects models were used to analyze changes in ratings on health anchoring vignettes and self-rated health over time.Main outcome measures: SRH at two time points with a -two-year span.Results: Both SRH and anchoring vignettes ratings displayed changes over a two-year span for all the studied age groups. Compared with the self-assessed change in health ("How would you rate your health as compared to that of last year?"), changes in SRH reported over time displayed a more stable and optimistic pattern. SRH responded to doctor diagnosed chronic disease and changes in functional limitation, before and after adjusting for evaluation standards.Conclusion: SRH is responsive to the newly diagnosed chronic disease and functional limitation, regardless of whether we consider response shift within the same respondents over time.

Poor reporting of physical activity and exercise interventions in youth mental health trials: A brief report

AIM

To describe the quality and completeness of the description and reporting of physical activity and exercise interventions delivered to young people to promote mental health or treat mental illness.

METHODS

We conducted a series of scoping reviews identifying 64 controlled trials of physical activity and exercise interventions delivered to young people. We extracted: intervention characteristics, personnel and delivery format, the intensity, duration, frequency and type of physical activity or exercise.

RESULTS

There was limited reporting of intervention details across studies; 52% did not provide information to confidently assess intervention intensity, 29% did not state who delivered the intervention, and 44% did not specify the intervention delivery format.

CONCLUSIONS

We recommend that authors adhere to the CONSORT reporting requirements and its intervention reporting extensions, (a) the Template for Intervention Description and Replication, (b) Consensus for Exercise Reporting Template and (c) as part of this, detail the frequency, intensity, time and type of physical activity recommendations and prescriptions. Without this, future trials are unable to replicate and extend previous work to support or disconfirm existing knowledge, leading to research waste and diminishing translation and implementation potential.

Role of standardized reporting and novel imaging markers in chronic pancreatitis

PURPOSE OF REVIEW

This article reviews recent efforts about standardized imaging features and reporting of chronic pancreatitis and recently published or ongoing imaging studies, which aim to establish novel imaging biomarkers for detection of parenchymal changes seen in chronic pancreatitis.

RECENT FINDINGS

New novel MRI techniques are being developed to increase the diagnostic yield of chronic pancreatitis specifically in the early stage. T1 relaxation time, T1 signal intensity ratio and extracellular volume fraction offer potential advantages over conventional cross-sectional imaging, including simplicity of analysis and more objective interpretation of observations allowing population-based comparisons. In addition, standardized definitions and reporting guidelines for chronic pancreatitis based on available evidence and expert consensus have been proposed. These new imaging biomarkers and reporting guidelines are being validated for prognostic/therapeutic assessment of adult patients participating in longitudinal studies of The Consortium for the Study of Chronic Pancreatitis, Diabetes and Pancreatic Cancer.

SUMMARY

New imaging biomarkers derived from novel MRI sequences promise a new chapter for diagnosis and severity assessment of chronic pancreatitis; a cross-sectional imaging-based diagnostic criteria for chronic pancreatitis combining ductal and parenchymal findings. Standardized imaging findings and reporting guidelines of chronic pancreatitis would enhance longitudinal assessment of disease severity in clinical trials and improve communication between radiologists and pancreatologists in clinical practice.

Conflicting information from the Food and Drug Administration: Missed opportunity to lead standards for safe and effective medical artificial intelligence solutions

The Food & Drug Administration (FDA) is considering the permanent exemption of premarket notification requirements for several Class I and II medical device products, including several artificial Intelligence (AI)-driven devices. The exemption is based on the need to rapidly more quickly disseminate devices to the public, estimated cost-savings, a lack of documented adverse events reported to the FDA's database. However, this ignores emerging issues related to AI-based devices, including utility, reproducibility and bias that may not only affect an individual but entire populations. We urge the FDA to reinforce the messaging on safety and effectiveness regulations of AI-based Software as a Medical Device products to better promote fair AI-driven clinical decision tools and for preventing harm to the patients we serve.

MINIMAR (MINimum Information for Medical AI Reporting): Developing reporting standards for artificial intelligence in health care

The rise of digital data and computing power have contributed to significant advancements in artificial intelligence (AI), leading to the use of classification and prediction models in health care to enhance clinical decision-making for diagnosis, treatment and prognosis. However, such advances are limited by the lack of reporting standards for the data used to develop those models, the model architecture, and the model evaluation and validation processes. Here, we present MINIMAR (MINimum Information for Medical AI Reporting), a proposal describing the minimum information necessary to understand intended predictions, target populations, and hidden biases, and the ability to generalize these emerging technologies. We call for a standard to accurately and responsibly report on AI in health care. This will facilitate the design and implementation of these models and promote the development and use of associated clinical decision support tools, as well as manage concerns regarding accuracy and bias.

The need for polygenic score reporting standards in evidence-based practice: lipid genetics use case

PURPOSE OF REVIEW

Polygenic scores (PGS) are used to quantify the genetic predisposition for heritable traits, with hypothesized utility for personalized risk assessments. Lipid PGS are primed for clinical translation, but evidence-based practice changes will require rigorous PGS standards to ensure reproducibility and generalizability. Here we review applicable reporting and technical standards for dyslipidemia PGS translation along phases of the ACCE (Analytical validity, Clinical validity, Clinical utility, Ethical considerations) framework for evaluating genetic tests.

RECENT FINDINGS

New guidance suggests existing standards for study designs incorporating the ACCE framework are applicable to PGS and should be adopted. One recent example is the Clinical Genomics Resource (ClinGen) and Polygenic Score Catalog's PRS reporting standards, which define minimal requirements for describing rationale for score development, study population definitions and data parameters, risk model development and application, risk model evaluation, and translational considerations, such as generalizability beyond the target population studied.

SUMMARY

Lipid PGS are likely to be integrated into clinical practice in the future. Clinicians will need to be prepared to determine if and when lipid PGS is useful and valid. This decision-making will depend on the quality of evidence for the clinical use of PGS. Establishing reporting standards for PGS will help facilitate data sharing and transparency for critical evaluation, ultimately benefiting the efficiency of evidence-based practice.

What Every Reader Should Know About Studies Using Electronic Health Record Data but May Be Afraid to Ask

Coincident with the tsunami of COVID-19–related publications, there has been a surge of studies using real-world data, including those obtained from the electronic health record (EHR). Unfortunately, several of these high-profile publications were retracted because of concerns regarding the soundness and quality of the studies and the EHR data they purported to analyze. These retractions highlight that although a small community of EHR informatics experts can readily identify strengths and flaws in EHR-derived studies, many medical editorial teams and otherwise sophisticated medical readers lack the framework to fully critically appraise these studies. In addition, conventional statistical analyses cannot overcome the need for an understanding of the opportunities and limitations of EHR-derived studies. We distill here from the broader informatics literature six key considerations that are crucial for appraising studies utilizing EHR data: data completeness, data collection and handling (eg, transformation), data type (ie, codified, textual), robustness of methods against EHR variability (within and across institutions, countries, and time), transparency of data and analytic code, and the multidisciplinary approach. These considerations will inform researchers, clinicians, and other stakeholders as to the recommended best practices in reviewing manuscripts, grants, and other outputs from EHR-data derived studies, and thereby promote and foster rigor, quality, and reliability of this rapidly growing field.

A systematic review of reporting quality for anaesthetic interventions in randomised controlled trials

Interventions from randomised controlled trials can only be replicated if they are reported in sufficient detail. The results of trials can only be confidently interpreted if the delivery of the intervention was systematic and the protocol adhered to. We systematically reviewed trials of anaesthetic interventions published in 12 journals from January 2016 to September 2019. We assessed the detail with which interventions were reported, using the Consolidated Standards of Reporting Trials statement for non‐pharmacological treatments. We analysed 162 interventions reported by 78 trials in 18,675 participants. Detail sufficiently precise to replicate the intervention was reported for 111 (69%) interventions. Intervention standardisation was reported for 135 (83%) out of the 162 interventions, and protocol adherence was reported for 20 (12%) interventions. Sixty (77%) out of the 78 trials reported the administrative context in which interventions were delivered and 36 (46%) trials detailed the expertise of the practitioners. We conclude that bespoke reporting tools should be developed for anaesthetic interventions and interventions in other areas such as critical care.

"statcheck": Automatically detect statistical reporting inconsistencies to increase reproducibility of meta-analyses

We present the R package and web app statcheck to automatically detect statistical reporting inconsistencies in primary studies and meta-analyses. Previous research has shown a high prevalence of reported p-values that are inconsistent - meaning a re-calculated p-value, based on the reported test statistic and degrees of freedom, does not match the author-reported p-value. Such inconsistencies affect the reproducibility and evidential value of published findings. The tool statcheck can help researchers to identify statistical inconsistencies so that they may correct them. In this paper, we provide an overview of the prevalence and consequences of statistical reporting inconsistencies. We also discuss the tool statcheck in more detail and give an example of how it can be used in a meta-analysis. We end with some recommendations concerning the use of statcheck in meta-analyses and make a case for better reporting standards of statistical results.

Imaging methods are vastly underreported in biomedical research

A variety of microscopy techniques are used by researchers in the life and biomedical sciences. As these techniques become more powerful and more complex, it is vital that scientific articles containing images obtained with advanced microscopes include full details about how each image was obtained. To explore the reporting of such details we examined 240 original research articles published in eight journals. We found that the quality of reporting was poor, with some articles containing no information about how images were obtained, and many articles lacking important basic details. Efforts by researchers, funding agencies, journals, equipment manufacturers and staff at shared imaging facilities are required to improve the reporting of experiments that rely on microscopy techniques.

Reporting Bayesian Results

Because of the different philosophy of Bayesian statistics, where parameters are random variables and data are considered fixed, the analysis and presentation of results will differ from that of frequentist statistics. Most importantly, the probabilities that a parameter is in certain regions of the parameter space are crucial quantities in Bayesian statistics that are not calculable (or considered important) in the frequentist approach that is the basis of much of traditional statistics. In this article, I discuss the implications of these differences for presentation of the results of Bayesian analyses. In doing so, I present more detailed guidelines than are usually provided and explain the rationale for my suggestions.

Use of the Template for Intervention Description and Replication (Tidier) Checklist in Social Work Research

Social work has a longstanding commitment to sound research and the development and dissemination of evidence-based practice. To that end, multiple professional groups have developed or refined guidelines for reporting research procedures and findings, with the objectives of enhancing transparency, integrity, and rigor in science. Such guidelines can also facilitate replication and systematic review. The Template for Intervention Description and Replication (TIDieR) checklist represents the culmination of a multi-stage process to expand upon existing reporting guidelines. As such, the checklist provides a framework for more transparent communication about empirically-grounded interventions addressing a broad range of social and behavioral health issues. Use of this checklist can be beneficial for researchers, practitioners, and recipients of social work interventions. After discussing selected background regarding the need for and benefit of reporting standards and describing the TIDieR measure, we outline practical considerations in the checklist's use by those engaged in social work research.

The "realist search": A systematic scoping review of current practice and reporting

The requirement for literature searches that identify studies for inclusion in systematic reviews should be systematic, explicit, and reproducible extends, at least by implication, to other types of literature review. However, realist reviews commonly require literature searches that challenge systematic reporting; searches are iterative and involve multiple search strategies and approaches. Notwithstanding these challenges, reporting of the "realist search" can be structured to be transparent and to facilitate identification of innovative retrieval practices. Our six-component search framework consolidates and extends the structure advanced by Pawson, one of the originators of realist review: formulating the question, conducting the background search, searching for program theory, searching for empirical studies, searching to refine program theory and identifying relevant mid-range theory, and documenting and reporting the search process. This study reviews reports of search methods in 34 realist reviews published within the calendar year of 2016. Data from all eligible reviews were extracted from the search framework. Realist search reports poorly differentiate between the different search components. Review teams often conduct a single "big bang" multipurpose search to fulfill multiple functions within the review. However, it is acknowledged that realist searches are likely to be iterative and responsive to emergent data. Overall, the search for empirical studies appears most comprehensive in conduct and reporting detail. In contrast, searches to identify and refine program theory are poorly conducted, if at all, and poorly reported. Use of this framework offers greater transparency in conduct and reporting while preserving flexibility and methodological innovation.

Inadequate description of placebo and sham controls in a systematic review of recent trials

BACKGROUND

Poorly described placebo/sham controls make it difficult to appraise active intervention benefits and harms. The 12-item Template for Intervention Description and Replication (TIDieR) checklist was developed to improve the reporting of active interventions. The extent to which TIDieR has been used to improve description of placebo or sham control is not known.

MATERIALS AND METHODS

We systematically identified and examined all placebo/sham-controlled randomised trials published in 2018 in the top six general medical journals. We reported how many of the TIDieR checklist items were used to describe the placebo/sham control(s). We supplemented this with a sample of 100 placebo/sham-controlled trials from any journal and searched Google Scholar to identify placebo/sham-controlled trials citing TIDieR.

RESULTS

We identified 94 placebo/sham-controlled trials published in the top journals in 2018. None reported using TIDieR, and none reported placebo or sham components completely. On average eight TIDieR items were addressed, with placebo/sham control name (100%) and when and how much was administered (97.9%) most commonly reported. Some items (rationale, 8.5%, whether there were modifications, 25.5%) were less often reported. In our sample of less well-cited journals, reporting was poorer (average of six items) and followed a similar pattern. Since TIDieR's first publication, six placebo-controlled trials have cited it according to Google Scholar. Two of these used the checklist to describe placebo controls; neither one completely desribed the placebo intervention.

CONCLUSIONS

Placebo and sham controls are poorly described within randomised trials, and TIDieR is rarely used to guide these descriptions. We recommend developing guidelines to promote better descriptions of placebo/sham control components within clinical trials.

Assessing the Completeness of Reporting in Preclinical Oncolytic Virus Therapy Studies

Irreproducibility of preclinical findings could be a significant barrier to the "bench-to-bedside" development of oncolytic viruses (OVs). A contributing factor is the incomplete and non-transparent reporting of study methodology and design. Using the NIH Principles and Guidelines for Reporting Preclinical Research, a core set of seven recommendations, we evaluated the completeness of reporting of preclinical OV studies. We also developed an evidence map identifying the current trends in OV research. A systematic search of MEDLINE and Embase identified all relevant articles published over an 18 month period. We screened 1,554 articles, and 236 met our a priori-defined inclusion criteria. Adenovirus (43%) was the most commonly used viral platform. Frequently investigated cancers included colorectal (14%), skin (12%), and breast (11%). Xenograft implantation (61%) in mice (96%) was the most common animal model. The use of preclinical reporting guidelines was listed in 0.4% of articles. Biological and technical replicates were completely reported in 1% of studies, statistics in 49%, randomization in 1%, blinding in 2%, sample size estimation in 0%, and inclusion/exclusion criteria in 0%. Overall, completeness of reporting in the preclinical OV therapy literature is poor. This may hinder efforts to interpret, replicate, and ultimately translate promising preclinical OV findings.

Extending the RIGHT statement for reporting adapted practice guidelines in healthcare: the RIGHT-Ad@pt Checklist protocol

INTRODUCTION

The adaptation of guidelines is an increasingly used methodology for the efficient development of contextualised recommendations. Nevertheless, there is no specific reporting guidance. The essential Reporting Items of Practice Guidelines in Healthcare (RIGHT) statement could be useful for reporting adapted guidelines, but it does not address all the important aspects of the adaptation process. The objective of our project is to develop an extension of the RIGHT statement for the reporting of adapted guidelines (RIGHT-Ad@pt Checklist).

METHODS AND ANALYSIS

To develop the RIGHT-Ad@pt Checklist, we will use a multistep process that includes: (1) establishment of a Working Group; (2) generation of an initial checklist based on the RIGHT statement; (3) optimisation of the checklist (an initial assessment of adapted guidelines, semistructured interviews, a Delphi consensus survey, an external review by guideline developers and users and a final assessment of adapted guidelines); and (4) approval of the final checklist. At each step of the process, we will calculate absolute frequencies and proportions, use content analysis to summarise and draw conclusions, discuss the results, draft a report and refine the checklist.

ETHICS AND DISSEMINATION

We have obtained a waiver of approval from the Clinical Research Ethics Committee at the Hospital de la Santa Creu i Sant Pau (Barcelona, Spain). We will disseminate the RIGHT-Ad@pt Checklist by publishing into a peer-reviewed journal, presenting to relevant stakeholders and translating into different languages. We will continuously seek feedback from stakeholders, surveil new relevant evidence and, if necessary, update the checklist.

Adherence Reporting in Randomized Controlled Trials Examining Manualized Multisession Online Interventions: Systematic Review of Practices and Proposal for Reporting Standards

Background

Adherence reflects the extent to which individuals experience or engage with the content of online interventions and poses a major challenge. Neglecting to examine and report adherence and its relation to outcomes can compromise the interpretation of research findings.

Objective

The aim of this systematic review is to analyze how adherence is accounted for in publications and to propose standards for measuring and reporting adherence to online interventions.

Methods

We performed a systematic review of randomized controlled trials on online interventions for the prevention and treatment of common mental disorders (depression, anxiety disorders, substance related disorders, and eating disorders) published between January 2006 and May 2018 and indexed in Medline and Web of Science. We included primary publications on manualized online treatments (more than 1 session and successive access to content) and examined how adherence was reported in these publications.

Results

We identified 216 publications that met our inclusion criteria. Adherence was addressed in 85% of full-text manuscripts, but only in 31% of abstracts. A median of three usage metrics were reported; the most frequently reported usage metric (61%) was intervention completion. Manuscripts published in specialized electronic health journals more frequently included information on the relation of adherence and outcomes.

Conclusions

We found substantial variety in the reporting of adherence and the usage metrics used to operationalize adherence. This limits the comparability of results and impedes the integration of findings from different studies. Based on our findings, we propose reporting standards for future publications on online interventions.

[Reporting standards for expert consensus on clinical practice of Chinese patent medicines of China Association of Chinese Medicine]

In 2018,Standardization Department of China Association of Chinese Medicine invited methodologists with the background of evidence-based medicine to discuss and draft a series of standards for expert consensus on clinical practice of Chinese patent medicines. These standards have been made by reference to the published standards for developing expert consensus and clinical practice guidelines. These standards were made based on full consideration of the current evidence status and the history of clinical practice of Chinese patent medicines. These standards were composed of four parts,namely information summary items,normative general items,normative technical items and information supplementary items,including cover,content,preface,introduction,title,scope,basic Information of Chinese patent medicine,suggestions for clinical application,safety,conflict of interest,appendix,and references,so as to provides reference for improving the quality of expert consensus-based compilation and enhancing the applicability of expert consensus.

Harnessing Energies, Resolving Tensions: Acknowledging a Dual Heritage for Qualitative Evidence Synthesis

Qualitative evidence synthesis (QES) encompasses more than 20 methods for synthesizing qualitative accounts of research phenomena documenting real-life contexts. However, tensions frequently arise from the different heritages that shape QES methodology: namely, systematic reviews of effectiveness and primary qualitative research. Methodological innovations either derive from each heritage or are stimulated when both are in juxtaposition; it is important to broker a rapprochement. This article draws on practical experience from a range of syntheses and methodological development work conducted with the Cochrane Qualitative and Implementation Methods Group. The legacy of both heritages is briefly characterized. Three stages of the QES process offer exemplars: searching/sampling, quality assessment, and data synthesis. Rather than an antagonistic clash of research paradigms, this dual heritage offers an opportunity to harness the collective energies of both paradigms. Future methodological research is needed to identify further applications by which this dual heritage might be optimally harnessed.

Using the Value of Information to improve conservation decision making

Conservation decisions are challenging, not only because they often involve difficult conflicts among outcomes that people value, but because our understanding of the natural world and our effects on it is fraught with uncertainty. Value of Information (VoI) methods provide an approach for understanding and managing uncertainty from the standpoint of the decision maker. These methods are commonly used in other fields (e.g. economics, public health) and are increasingly used in biodiversity conservation. This decision-analytical approach can identify the best management alternative to select where the effectiveness of interventions is uncertain, and can help to decide when to act and when to delay action until after further research. We review the use of VoI in the environmental domain, reflect on the need for greater uptake of VoI, particularly for strategic conservation planning, and suggest promising areas for new research. We also suggest common reporting standards as a means of increasing the leverage of this powerful tool. The environmental science, ecology and biodiversity categories of the Web of Knowledge were searched using the terms 'Value of Information,' 'Expected Value of Perfect Information,' and the abbreviation 'EVPI.' Google Scholar was searched with the same terms, and additionally the terms decision and biology, biodiversity conservation, fish, or ecology. We identified 1225 papers from these searches. Included studies were limited to those that showed an application of VoI in biodiversity conservation rather than simply describing the method. All examples of use of VOI were summarised regarding the application of VoI, the management objectives, the uncertainties, the models used, how the objectives were measured, and the type of VoI. While the use of VoI appears to be on the increase in biodiversity conservation, the reporting of results is highly variable, which can make it difficult to understand the decision context and which uncertainties were considered. Moreover, it was unclear if, and how, the papers informed management and policy interventions, which is why we suggest a range of reporting standards that would aid the use of VoI. The use of VoI in conservation settings is at an early stage. There are opportunities for broader applications, not only for species-focussed management problems, but also for setting local or global research priorities for biodiversity conservation, making funding decisions, or designing or improving protected area networks and management. The long-term benefits of applying VoI methods to biodiversity conservation include a more structured and decision-focused allocation of resources to research.

Scientific Reporting: Raising the Standards

This article is based on a presentation that was made at the 2014 annual meeting of the editorial board of Health Education & Behavior. The article addresses critical issues related to standards of scientific reporting in journals, including concerns about external and internal validity and reporting bias. It reviews current reporting guidelines, effects of adopting guidelines, and offers suggestions for improving reporting. The evidence about the effects of guideline adoption and implementation is briefly reviewed. Recommendations for adoption and implementation of appropriate guidelines, including considerations for journals, are provided.

On the Diagnosis of Mycotic Aortic Aneurysms

Objective:

There is striking paucity in consensus on the terminology, definition, and diagnostic criteria of mycotic aortic aneurysms. This literature study aims to elucidate this scientific omission, discuss its consequences, and present a proposition for reporting items on this disease.

Methods:

A systematic literature review on PubMed and Medline using mycotic and infected aortic aneurysms between 1850 and 2017 was performed. Articles were assessed according to a protocol regarding terminology, definition, and diagnostic criteria. Case series with less than 5 patients were excluded.

Results:

A total of 49 articles were included. The most prevalent term was mycotic aortic aneurysm but there was no widely accepted definition. Most modern publications used a diagnostic workup based on a combination on clinical presentation, laboratory results, imaging findings, and intraoperative findings. How these protean variables should be balanced was unclear. A proposition of reporting items was framed and consisted of definition of disease used, basis of diagnostic workup, exclusion criteria, patient characteristics, laboratory and imaging findings, aneurysm anatomy, details on treatment, pre/postoperative antibiotic treatment, and details on follow-up.

Conclusions:

This article emphasizes the need to standardize definition, terminology, and diagnostic criteria for mycotic aortic aneurysms and proposes reporting items enhancing comparability between studies.

CJCheck Stage 1: development and testing of a checklist for reporting community juries - Delphi process and analysis of studies published in 1996-2015

Background

Opportunities for community members to actively participate in policy development are increasing. Community/citizen's juries (CJs) are a deliberative democratic process aimed to illicit informed community perspectives on difficult topics. But how comprehensive these processes are reported in peer‐reviewed literature is unknown. Adequate reporting of methodology enables others to judge process quality, compare outcomes, facilitate critical reflection and potentially repeat a process. We aimed to identify important elements for reporting CJs, to develop an initial checklist and to review published health and health policy CJs to examine reporting standards.

Design

Using the literature and expertise from CJ researchers and policy advisors, a list of important CJ reporting items was suggested and further refined. We then reviewed published CJs within the health literature and used the checklist to assess the comprehensiveness of reporting.

Results

CJCheck was developed and examined reporting of CJ planning, juror information, procedures and scheduling. We screened 1711 studies and extracted data from 38. No studies fully reported the checklist items. The item most consistently reported was juror numbers (92%, 35/38), while least reported was the availability of expert presentations (5%, 2/38). Recruitment strategies were described in 66% of studies (25/38); however, the frequency and timing of deliberations was inadequately described (29%, 11/38).

Conclusions

Currently CJ publications in health and health policy literature are inadequately reported, hampering their use in policy making. We propose broadening the CJCheck by creating a reporting standards template in collaboration with international CJ researchers, policy advisors and consumer representatives to ensure standardized, systematic and transparent reporting.

Standard of reporting animal-based experimental research in Indian Journal of Pharmacology

Objective:

The objective of present study was to survey and determine the reporting standards of animal studies published during three years from 2012 to 2014 in the Indian Journal of Pharmacology (IJP).

Material and Methods:

All issues of IJP published in the year 2012, 2013 and 2014 were reviewed to identify animal studies. Each animal study was searched for 15 parameters specifically designed to review standards of animal experimentation and research methodology.

Observation:

All published studies had clearly defined aims and objectives while a statement on ethical clearance about the study protocol was provided in 97% of papers. Information about animal strain and sex was given in 91.8% and 90% of papers respectively. Age of experimental animals was mentioned by 44.4% papers while source of animals was given in 50.8% papers. Randomization was reported by 37.4% while 9.9% studies reported blinding. Only 3.5% studies mentioned any limitations of their work.

Conclusion:

Present study demonstrates relatively good reporting standards in animal studies published in IJP. The items which need to be improved are randomization, blinding, sample size calculation, stating the limitations of study, sources of support and conflict of interest. The knowledge shared in the present paper could be used for better reporting of animal based experiments.

A Framework for Evaluating and Enhancing Alignment in Self-Regulated Learning Research

We discuss the articles of this special issue with reference to an important yet previously only implicit dimension of study quality: alignment across the theoretical and methodological decisions that collectively define an approach to self-regulated learning. Integrating and extending work by leaders in the field, we propose a framework for evaluating alignment in the way self-regulated learning research is both conducted and reported. Within this framework, the special issue articles provide a springboard for discussing methodological promises and pitfalls of increasingly sophisticated research on the dynamic, contingent, and contextualized features of self-regulated learning.

Developing standards for reporting implementation studies of complex interventions (StaRI): a systematic review and e-Delphi

BACKGROUND

Dissemination and implementation of health care interventions are currently hampered by the variable quality of reporting of implementation research. Reporting of other study types has been improved by the introduction of reporting standards (e.g. CONSORT). We are therefore developing guidelines for reporting implementation studies (StaRI).

METHODS

Using established methodology for developing health research reporting guidelines, we systematically reviewed the literature to generate items for a checklist of reporting standards. We then recruited an international, multidisciplinary panel for an e-Delphi consensus-building exercise which comprised an initial open round to revise/suggest a list of potential items for scoring in the subsequent two scoring rounds (scale 1 to 9). Consensus was defined a priori as 80% agreement with the priority scores of 7, 8, or 9.

RESULTS

We identified eight papers from the literature review from which we derived 36 potential items. We recruited 23 experts to the e-Delphi panel. Open round comments resulted in revisions, and 47 items went forward to the scoring rounds. Thirty-five items achieved consensus: 19 achieved 100% agreement. Prioritised items addressed the need to: provide an evidence-based justification for implementation; describe the setting, professional/service requirements, eligible population and intervention in detail; measure process and clinical outcomes at population level (using routine data); report impact on health care resources; describe local adaptations to the implementation strategy and describe barriers/facilitators. Over-arching themes from the free-text comments included balancing the need for detailed descriptions of interventions with publishing constraints, addressing the dual aims of reporting on the process of implementation and effectiveness of the intervention and monitoring fidelity to an intervention whilst encouraging adaptation to suit diverse local contexts.

CONCLUSIONS

We have identified priority items for reporting implementation studies and key issues for further discussion. An international, multidisciplinary workshop, where participants will debate the issues raised, clarify specific items and develop StaRI standards that fit within the suite of EQUATOR reporting guidelines, is planned.

REGISTRATION

The protocol is registered with Equator: http://www.equator-network.org/library/reporting-guidelines-under-development/#17 .

Publishing protocols for partnered research

Published scientific protocols are advocated as a means of controlling bias in research reporting. Indeed, many journals require a study protocol with manuscript submission. However, publishing protocols of partnered research (PPR) can be challenging in light of the research model's dynamic nature, especially as no current reporting standards exist. Nevertheless, as these protocols become more prevalent, a priori documentation of methods in partnered research studies becomes increasingly important. Using as illustration a suite of studies aimed at improving coordination and communication in the primary care setting, we sought to identify challenges in publishing PPR relative to traditional designs, present alternative solutions to PPR publication, and propose an initial checklist of content to be included in protocols of partnered research. Challenges to publishing PPR include reporting details of research components intended to be co-created with operational partners, changes to sampling and entry strategy, and alignment of scientific and operational goals. Proposed solutions include emulating reporting standards of qualitative research, participatory action research, and adaptive trial designs, as well as embracing technological tools that facilitate publishing adaptive protocols, with version histories that are able to be updated as major protocol changes occur. Finally, we present a proposed checklist of reporting elements for partnered research protocols.

A comprehensive database of verified experimental data on protein folding kinetics

Insights into protein folding rely increasingly on the synergy between experimental and theoretical approaches. Developing successful computational models requires access to experimental data of sufficient quantity and high quality. We compiled folding rate constants for what initially appeared to be 184 proteins from 15 published collections/web databases. To generate the highest confidence in the dataset, we verified the reported lnkf value and exact experimental construct and conditions from the original experimental report(s). The resulting comprehensive database of 126 verified entries, ACPro, will serve as a freely accessible resource (https://www.ats.amherst.edu/protein/) for the protein folding community to enable confident testing of predictive models. In addition, we provide a streamlined submission form for researchers to add new folding kinetics results, requiring specification of all the relevant experimental information according to the standards proposed in 2005 by the protein folding consortium organized by Plaxco. As the number and diversity of proteins whose folding kinetics are studied expands, our curated database will enable efficient and confident incorporation of new experimental results into a standardized collection. This database will support a more robust symbiosis between experiment and theory, leading ultimately to more rapid and accurate insights into protein folding, stability, and dynamics.

Methods for documenting systematic review searches: a discussion of common issues

INTRODUCTION

As standardized reporting requirements for systematic reviews are being adopted more widely, review authors are under greater pressure to accurately record their search process. With careful planning, documentation to fulfill the Preferred Reporting Items for Systematic Reviews and Meta-Analyses requirements can become a valuable tool for organizing a systematic review literature search and planning updates.

METHODS

A working group of information specialists convened to discuss current practice and were informed by a Web-based survey of over 260 systematic review authors, trials search coordinators, librarians, and other information specialists conducted in February/March 2011.

DISCUSSION

Survey responses provided insight into current practices and difficulties of reporting searches. These included a lack of time, tools, clear understanding of the requirements, and uncertainty about responsibility for documenting these elements. This paper will present some of the practical aspects of documenting the systematic literature search. Section 1 provides background information and rationale for this paper. Section 2 discusses issues and recommendations arising from survey results. Section 3 outlines specific elements to be recorded. Section 4 guides the reader through the information management process. Section 5 concludes with implications for future research and practice. These principles are applicable to any large literature search for systematic reviews, health technology assessments, and guideline development.

Study flow diagrams in Cochrane systematic review updates: an adapted PRISMA flow diagram

Cochrane systematic reviews are conducted and reported according to rigorous standards. A study flow diagram must be included in a new review, and there is clear guidance from the PRISMA statement on how to do this. However, for a review update, there is currently no guidance on how study flow diagrams should be presented. To address this, a working group was formed to find a solution and produce guidance on how to use these diagrams in review updates.A number of different options were devised for how these flow diagrams could be used in review updates, and also in cases where multiple searches for a review or review update have been conducted. These options were circulated to the Cochrane information specialist community for consultation and feedback. Following the consultation period, the working group refined the guidance and made the recommendation that for review updates an adapted PRISMA flow diagram should be used, which includes an additional box with the number of previously included studies feeding into the total. Where multiple searches have been conducted, the results should be added together and treated as one set of results.There is no existing guidance for using study flow diagrams in review updates. Our adapted diagram is a simple and pragmatic solution for showing the flow of studies in review updates.

Developing a Reporting Guideline for Social and Psychological Intervention Trials

Social and psychological interventions are often complex. Understanding randomized controlled trials (RCTs) of these complex interventions requires a detailed description of the interventions tested and the methods used to evaluate them; however, RCT reports often omit, or inadequately report, this information. Incomplete and inaccurate reporting hinders the optimal use of research, wastes resources, and fails to meet ethical obligations to research participants and consumers. In this article, we explain how reporting guidelines have improved the quality of reports in medicine and describe the ongoing development of a new reporting guideline for RCTs: Consolidated Standards of Reporting Trials-SPI (an extension for social and psychological interventions). We invite readers to participate in the project by visiting our website, in order to help us reach the best-informed consensus on these guidelines (http://tinyurl.com/CONSORT-study).

Developing a reporting guideline for social and psychological intervention trials

Social and psychological interventions are often complex. Understanding randomised controlled trials (RCTs) of these complex interventions requires a detailed description of the interventions tested and the methods used to evaluate them; however, RCT reports often omit, or inadequately report, this information. Incomplete and inaccurate reporting hinders the optimal use of research, wastes resources, and fails to meet ethical obligations to research participants and consumers. In this paper, we explain how reporting guidelines have improved the quality of reports in medicine, and describe the ongoing development of a new reporting guideline for RCTs: CONSORT-SPI (an Extension for social and psychological interventions). We invite readers to participate in the project by visiting our website, in order to help us reach the best-informed consensus on these guidelines (http://tinyurl.com/CONSORT-study).