Data-sharing recommendations in biomedical journals and randomised controlled trials: an audit of journals following the ICMJE recommendations

Endorsement of open science practices by dental journals: A meta-research study

OBJECTIVES

This study evaluates the endorsement of open science practices by dental journals.

MATERIALS AND METHODS

This was a meta-research study that included journals listed in the 2021 Journal Citation Reports under Dentistry. A comprehensive evaluation was performed by accessing journal websites to ascertain the availability of publicly accessible instructions to authors in Portuguese, English, or Spanish. A researcher extracted information from the "Instructions for Authors" section, encompassing the journal's impact factor, mention of any reporting guidelines, details on data sharing, acceptance of articles in preprint format, and information regarding study protocol registration. Descriptive data analysis was conducted using the Stata 14.0 program, and an Open Science Score (OSS) (ranging from 0 to 100 %) was calculated for each journal by considering five open science practices. Pearson's correlation test was conducted to determine the relationship between the OSS score and journal impact factor.

RESULTS

Ninety journals were included in the study. Most journals (70 %) indicated the mandatory use of reporting guidelines, while 60 % recommended data sharing. Conversely, 46.7 % did not provide information on study protocol registration, and 44.4 % stipulated them as mandatory for authors. Regarding preprints, 50 % of the journals did not provide any information, but 46.7 % confirmed their acceptance. The mean OSS was 52.9 % (standard deviation 26.2). There was a weak correlation (Pearson's correlation coefficient of 0.221) between the journal impact factor and OSS (P-value=0.036).

CONCLUSION

This study found varying degrees of endorsement of open science practices among dental journals.

CLINICAL SIGNIFICANCE

Dental practitioners rely on high-quality, evidence-based research for informed decision-making. By assessing the endorsement of open science practices, our study contributes to improving the quality and reliability of dental research, ultimately enhancing the evidence base for clinical practice.

Cross-Sectional Evaluation of Open Science Practices at Imaging Journals: A Meta-Research Study

Objective: To evaluate open science policies of imaging journals, and compliance to these policies in published articles. Methods: From imaging journals listed we extracted open science policy details: protocol registration, reporting guidelines, funding, ethics and conflicts of interest (COI), data sharing, and open access publishing. The 10 most recently published studies from each journal were assessed to determine adherence to these policies. We calculated the proportion of open science policies into an Open Science Score (OSS) for all journals and articles. We evaluated relationships between OSS and journal/article level variables. Results: 82 journals/820 articles were included. The OSS of journals and articles was 58.3% and 31.8%, respectively. Of the journals, 65.9% had registration and 78.1% had reporting guideline policies. 79.3% of journals were members of COPE, 81.7% had plagiarism policies, 100% required disclosure of funding, and 97.6% required disclosure of COI and ethics approval. 81.7% had data sharing policies and 15.9% were fully open access. 7.8% of articles had a registered protocol, 8.4% followed a reporting guideline, 77.4% disclosed funding, 88.7% disclosed COI, and 85.6% reported ethics approval. 12.3% of articles shared their data. 51% of articles were available through open access or as a preprint. OSS was higher for journal with DOAJ membership (80% vs 54.2%; P < .0001). Impact factor was not correlated with journal OSS. Knowledge synthesis articles has a higher OSS scores (44.5%) than prospective/retrospective studies (32.6%, 30.0%, P < .0001). Conclusion: Imaging journals endorsed just over half of open science practices considered; however, the application of these practices at the article level was lower.

Lifting of Embargoes to Data Sharing in Clinical Trials Published in Top Medical Journals

This study assesses data sharing status and lifting of embargoes in randomized clinical trials from top medical journals 3 to 5 years after publication.

Industry Involvement and Transparency in the Most Cited Clinical Trials, 2019-2022

Importance

Industry involvement is prominent in influential clinical trials, and commitments to transparency of trials are highly variable.

Objective

To evaluate the modes of industry involvement and the transparency features of the most cited recent clinical trials across medicine.

Design, Setting, and Participants

This cross-sectional study was a meta-research assessment including randomized and nonrandomized clinical trials published in 2019 or later. The 600 trials of any type of disease or setting that attracted highest number of citations in Scopus as of December 2022 were selected for analysis. Data were analyzed from March to September 2023.

Main Outcomes and Measures

Outcomes of interest were industry involvement (sponsor, author, and analyst) and transparency (protocols, statistical analysis plans, and data and code availability).

Results

Among 600 trials with a median (IQR) sample size of 415 (124-1046) participants assessed, 409 (68.2%) had industry funding and 303 (50.5%) were exclusively industry-funded. A total of 354 trials (59.0%) had industry authors, with 280 trials (46.6%) involving industry analysts and 125 trials (20.8%) analyzed exclusively by industry analysts. Among industry-funded trials, 364 (89.0%) reached conclusions favoring the sponsor. Most trials (478 trials [79.7%]) provided a data availability statement, and most indicated intention to share the data, but only 16 trials (2.7%) had data already readily available to others. More than three-quarters of trials had full protocols (482 trials [82.0%]) or statistical analysis plans (446 trials [74.3%]) available, but only 27 trials (4.5%) explicitly mentioned sharing analysis code (8 readily available; 19 on request). Randomized trials were more likely than nonrandomized studies to involve only industry analysts (107 trials [22.9%] vs 18 trials [13.6%]; P = .02) and to have full protocols (405 studies [86.5%] vs 87 studies [65.9%]; P < .001) and statistical analysis plans (373 studies [79.7%] vs 73 studies [55.3%]; P < .001) available. Almost all nonrandomized industry-funded studies (90 of 92 studies [97.8%]) favored the sponsor. Among industry-funded trials, exclusive industry funding (odds ratio, 2.9; 95% CI, 1.5-5.4) and industry-affiliated authors (odds ratio, 2.9; 95% CI, 1.5-5.6) were associated with favorable conclusions for the sponsor.

Conclusions and Relevance

This cross-sectional study illustrates how industry involvement in the most influential clinical trials was prominent not only for funding, but also authorship and provision of analysts and was associated with conclusions favoring the sponsor. While most influential trials reported that they planned to share data and make both protocols and statistical analysis plans available, raw data and code were rarely readily available.

Research Into Evidence-Based Psychological Interventions Needs a Stronger Focus on Replicability

Background

It is a precondition for evidence-based practice that research is replicable in a wide variety of clinical settings. Current standards for identifying evidence-based psychological interventions and making recommendations for clinical practice in clinical guidelines include criteria that are relevant for replicability, but a better understanding as well refined definitions of replicability are needed enabling empirical research on this topic. Recent advances on this issue were made in the wider field of psychology and in other disciplines, which offers the opportunity to define and potentially increase replicability also in research on psychological interventions.

Method

This article proposes a research strategy for assessing, understanding, and improving replicability in research on psychological interventions.

Results/Conclusion

First, we establish a replication taxonomy ranging from direct to conceptual replication adapted to the field of research on clinical interventions, propose study characteristics that increase the trustworthiness of results, and define statistical criteria for successful replication with respect to the quantitative outcomes of the original and replication studies. Second, we propose how to establish such standards for future research, i.e., in order to design future replication studies for psychological interventions as well as to apply them when investigating which factors are causing the (non-)replicability of findings in the current literature.

Prevalence and predictors of data and code sharing in the medical and health sciences: systematic review with meta-analysis of individual participant data

OBJECTIVES

To synthesise research investigating data and code sharing in medicine and health to establish an accurate representation of the prevalence of sharing, how this frequency has changed over time, and what factors influence availability.

DESIGN

Systematic review with meta-analysis of individual participant data.

DATA SOURCES

Ovid Medline, Ovid Embase, and the preprint servers medRxiv, bioRxiv, and MetaArXiv were searched from inception to 1 July 2021. Forward citation searches were also performed on 30 August 2022.

REVIEW METHODS

Meta-research studies that investigated data or code sharing across a sample of scientific articles presenting original medical and health research were identified. Two authors screened records, assessed the risk of bias, and extracted summary data from study reports when individual participant data could not be retrieved. Key outcomes of interest were the prevalence of statements that declared that data or code were publicly or privately available (declared availability) and the success rates of retrieving these products (actual availability). The associations between data and code availability and several factors (eg, journal policy, type of data, trial design, and human participants) were also examined. A two stage approach to meta-analysis of individual participant data was performed, with proportions and risk ratios pooled with the Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis.

RESULTS

The review included 105 meta-research studies examining 2 121 580 articles across 31 specialties. Eligible studies examined a median of 195 primary articles (interquartile range 113-475), with a median publication year of 2015 (interquartile range 2012-2018). Only eight studies (8%) were classified as having a low risk of bias. Meta-analyses showed a prevalence of declared and actual public data availability of 8% (95% confidence interval 5% to 11%) and 2% (1% to 3%), respectively, between 2016 and 2021. For public code sharing, both the prevalence of declared and actual availability were estimated to be <0.5% since 2016. Meta-regressions indicated that only declared public data sharing prevalence estimates have increased over time. Compliance with mandatory data sharing policies ranged from 0% to 100% across journals and varied by type of data. In contrast, success in privately obtaining data and code from authors historically ranged between 0% and 37% and 0% and 23%, respectively.

CONCLUSIONS

The review found that public code sharing was persistently low across medical research. Declarations of data sharing were also low, increasing over time, but did not always correspond to actual sharing of data. The effectiveness of mandatory data sharing policies varied substantially by journal and type of data, a finding that might be informative for policy makers when designing policies and allocating resources to audit compliance.

SYSTEMATIC REVIEW REGISTRATION

Open Science Framework doi:10.17605/OSF.IO/7SX8U.

Developing a standardized but extendable framework to increase the findability of infectious disease datasets

Biomedical datasets are increasing in size, stored in many repositories, and face challenges in FAIRness (findability, accessibility, interoperability, reusability). As a Consortium of infectious disease researchers from 15 Centers, we aim to adopt open science practices to promote transparency, encourage reproducibility, and accelerate research advances through data reuse. To improve FAIRness of our datasets and computational tools, we evaluated metadata standards across established biomedical data repositories. The vast majority do not adhere to a single standard, such as Schema.org, which is widely-adopted by generalist repositories. Consequently, datasets in these repositories are not findable in aggregation projects like Google Dataset Search. We alleviated this gap by creating a reusable metadata schema based on Schema.org and catalogued nearly 400 datasets and computational tools we collected. The approach is easily reusable to create schemas interoperable with community standards, but customized to a particular context. Our approach enabled data discovery, increased the reusability of datasets from a large research consortium, and accelerated research. Lastly, we discuss ongoing challenges with FAIRness beyond discoverability.

Clinical trial data-sharing policies among journals, funding agencies, foundations, and other professional organizations: a scoping review

BACKGROUND AND OBJECTIVES

To identify the similarities and differences in data-sharing policies for clinical trial data that are endorsed by biomedical journals, funding agencies, and other professional organizations. Additionally, to determine the beliefs, and opinions regarding data-sharing policies for clinical trials discussed in articles published in biomedical journals.

METHODS

Two searches were conducted, a bibliographic search for published articles that present beliefs, opinions, similarities, and differences regarding policies governing the sharing of clinical trial data. The second search analyzed the gray literature (non-peer-reviewed publications) to identify important data-sharing policies in selected biomedical journals, foundations, funding agencies, and other professional organizations.

RESULTS

A total of 471 articles were included after database search and screening, with 45 from the bibliographic search and 426 from the gray literature search. A total of 424 data-sharing policies were included. Fourteen of the 45 published articles from the bibliographic search (31.1%) discussed only advantages specific to data-sharing policies, 27 (27/45; 60%) discussed both advantages and disadvantages, and 4 (4/45; 8.9%) discussed only disadvantages specific. A total of 216 journals (of 270; 80%) specified a data-sharing policy provided by the journal itself. One hundred industry data-sharing policies were included, and 32 (32%) referenced a data-sharing policy on their website. One hundred and thirty-six (42%) organizations (of 327) specified a data-sharing policy.

CONCLUSION

We found many similarities listed as advantages to data-sharing and fewer disadvantages were discussed within the literature. Additionally, we found a wide variety of commonalities and differences-such as the lack of standardization between policies, and inadequately addressed details regarding the accessibility of research data-that exist in data-sharing policies endorsed by biomedical journals, funding agencies, and other professional organizations. Our study may not include information on all data sharing policies and our data is limited to the entities' descriptions of each policy.

Open science and conflict of interest policies of medical and health sciences journals before and during the COVID-19 pandemic: A repeat cross-sectional study

Objectives

To audit the transparent and open science standards of health and medical sciences journal policies and explore the impact of the COVID-19 pandemic.

Design

Repeat cross-sectional study.

Setting

19 journals listed in Google Scholar's Top Publications for health and medical sciences.

Participants

Blood, Cell, Circulation, European Heart Journal, Gastroenterology, Journal of Clinical Oncology, Journal of the American College of Cardiology, Nature Genetics, Nature Medicine, Nature Neuroscience, Neuron, PLoS ONE, Proceedings of the National Academy of Sciences, Science Translational Medicine, The British Medical Journal, The Journal of the American Medical Association, The Lancet, The Lancet Oncology, and The New England Journal of Medicine.

Main outcome measures

We used the Transparency and Openness Promotion (TOP) guideline and the International Committee of Medical Journal Editors (ICMJE) requirements for disclosing conflicts of interest (COIs) to evaluate journals standards.

Results

TOP scores slightly improved during the COVID-19 pandemic, from a median of 5 (IQR: 2–12.5) out of a possible 24 points in February 2020 to 7 (IQR: 4–12) in May 2021, but overall, scores were very low at both time points. Journal policies scored highest for their adherence to data transparency and scored lowest for preregistration of study protocols and analysis plans and the submission of replication studies. Most journals fulfilled all ICMJE provisions for reporting COIs before (84%; n  =  16) and during (95%; n  =  18) the COVID-19 pandemic.

Conclusions

The COVID-19 pandemic has highlighted the importance of practising open science. However, requirements for open science practices in audited policies were overall low, which may impede progress in health and medical research. As key stakeholders in disseminating research, journals should promote a research culture of greater transparency and more robust open science practices.

Sharing study materials in health and medical research

Making study materials available allows for a more comprehensive understanding of the scientific literature. Sharing can take many forms and include a wide variety of outputs including code and data. Biomedical research can benefit from increased transparency but faces unique challenges for sharing, for instance, confidentiality concerns around participants' medical data. Both general and specialised repositories exist to aid in sharing most study materials. Sharing may also require skills and resources to ensure that it is done safely and effectively. Educating researchers on how to best share their materials, and properly rewarding these practices, requires action from a variety of stakeholders including journals, funders and research institutions.

Data Sharing and Reanalyses Among Randomized Clinical Trials Published in Surgical Journals Before and After Adoption of a Data Availability and Reproducibility Policy

Importance

Clinical trial data sharing holds promise for maximizing the value of clinical research. The International Committee of Medical Journal Editors (ICMJE) adopted a policy promoting data sharing in July 2018.

Objective

To evaluate the association of the ICMJE data sharing policy with data availability and reproducibility of main conclusions among leading surgical journals.

Design, Setting, and Participants

This cross-sectional study, conducted in October 2021, examined randomized clinical trials (RCTs) in 10 leading surgical journals before and after the implementation of the ICMJE data sharing policy in July 2018.

Exposure

Implementation of the ICMJE data sharing policy.

Main Outcomes and Measures

To demonstrate a pre-post increase in data availability from 5% to 25% (α = .05; β = 0.1), 65 RCTs published before and 65 RCTs published after the policy was issued were included, and their data were requested. The primary outcome was data availability (ie, the receipt of sufficient data to enable reanalysis of the primary outcome). When data sharing was available, the primary outcomes reported in the journal articles were reanalyzed to explore reproducibility. The reproducibility features of these studies were detailed.

Results

Data were available for 2 of 65 RCTs (3.1%) published before the ICMJE policy and for 2 of 65 RCTs (3.1%) published after the policy was issued (odds ratio, 1.00; 95% CI, 0.07-14.19; P > .99). A data sharing statement was observed in 11 of 65 RCTs (16.9%) published after the policy vs none before the policy (risk ratio, 2.20; 95% CI, 1.81-2.68; P = .001). Data obtained for reanalysis (n = 4) were not from RCTs published with a data sharing statement. Of the 4 RCTs with available data, all of them had primary outcomes that were fully reproduced. However, discrepancies or inaccuracies that were not associated with study conclusions were identified in 3 RCTs. These concerned the number of patients included in 1 RCT, the management of missing values in another RCT, and discrepant timing for the principal outcome declared in the study registration and reported in the third RCT.

Conclusions and Relevance

This cross-sectional study suggests that data sharing practices are rare in surgical journals despite the ICMJE policy and that most RCTs published in these journals lack transparency. The results of these studies may not be reproducible by external researchers.

Reporting and data sharing level for COVID-19 vaccine trials: A cross-sectional study

BACKGROUND

The results and data availability of vaccine trials directly affect the decisions of healthcare providers, the public, and policymakers as to whether the vaccine should be applied. However, the reporting and data sharing level of COVID-19 vaccine studies are not clear.

METHODS

A cross-sectional study was conducted. A systematic search up to 9 May 2021 in 12 databases and an updated search to 6 July 2021 were conducted in the Cochrane Living Systematic Review and Network Meta-Analysis database to identify COVID-19 vaccine trials. The basic characteristics of included trials were summarized. The reporting level was assessed according to the CONSORT checklist. The data sharing level was assessed by open science practices. Types of incomplete reporting including protocol deviation, lack of primary outcomes clarity, and the omission of harms were analyzed.

FINDINGS

Finally, thirty-six COVID-19 vaccine articles reporting on 40 randomized controlled trials were included in this analysis. Based on the CONSORT checklist, the mean reporting score was 29.7 [95% confidence interval 28.7, 30.7]. Thirty-one articles (31/36, 86.1%) had data sharing statements, twenty-five articles (25/36, 69.4%) provided access to the source data. Twenty-seven articles (27/36, 75.0%) had protocol deviation, lack of primary outcomes clarity, or the omission of harms.

INTERPRETATION

The reporting and data sharing level of COVID-19 vaccine trials were not optimal. We hope that the reporting and data sharing of future trials will be improved. We recommend establishing a comprehensive, accurate data sharing system for future vaccine trials.

FUNDING

This work was supported by the National Key R&D Program of China (2019YFC1710400; 2019YFC1710403).

Methodology over metrics: current scientific standards are a disservice to patients and society

Covid-19 research made it painfully clear that the scandal of poor medical research, as denounced by Altman in 1994, persists today. The overall quality of medical research remains poor, despite longstanding criticisms. The problems are well known, but the research community fails to properly address them. We suggest that most problems stem from an underlying paradox: although methodology is undeniably the backbone of high-quality and responsible research, science consistently undervalues methodology. The focus remains more on the destination (research claims and metrics) than on the journey. Notwithstanding, research should serve society more than the reputation of those involved. While we notice that many initiatives are being established to improve components of the research cycle, these initiatives are too disjointed. The overall system is monolithic and slow to adapt. We assert that top-down action is needed from journals, universities, funders and governments to break the cycle and put methodology first. These actions should involve the widespread adoption of registered reports, balanced research funding between innovative, incremental and methodological research projects, full recognition and demystification of peer review, improved methodological review of reports, adherence to reporting guidelines, and investment in methodological education and research. Currently, the scientific enterprise is doing a major disservice to patients and society.

Data-sharing practices in publications funded by the Canadian Institutes of Health Research: a descriptive analysis

BACKGROUND

As Canada increases requirements for research data management and sharing, there is value in identifying how research data are shared and what has been done to make them findable and reusable. This study aimed to understand Canada's data-sharing landscape by reviewing how data funded by the Canadian Institutes of Health Research (CIHR) are shared and comparing researchers' data-sharing practices to best practices for research data management and sharing.

METHODS

We performed a descriptive analysis of CIHR-funded publications from PubMed and PubMed Central published between 1946 and Dec. 31, 2019, that indicated that the research data underlying the results of the publication were shared. We analyzed each publication to identify how and where data were shared, who shared data and what documentation was included to support data reuse.

RESULTS

Of 4144 CIHR-funded publications identified, 1876 (45.2%) included accessible data, 935 (22.6%) stated that data were available via request or application, and 300 (7.2%) stated that data sharing was not applicable or possible; we found no evidence of data sharing in 1558 publications (37.6%). Frequent data-sharing methods included via a repository (1549 [37.4%]), within supplementary files (1048 [25.3%]) and via request or application (935 [22.6%]). Overall, 554 publications (13.4%) included documentation that would facilitate data reuse.

INTERPRETATION

Publications funded by the CIHR largely lack the metadata, access instructions and documentation to facilitate data discovery and reuse. Without measures to address these concerns and enhanced support for researchers seeking to implement best practices for research data management and sharing, much CIHR-funded research data will remain hidden, inaccessible and unusable.

Medical journal requirements for clinical trial data sharing: Ripe for improvement

Florian Naudet and co-authors discuss strengthening requirements for sharing clinical trial data.

COVID-19 and the research scholarship ecosystem: help!

OBJECTIVES

Data sharing practices remain elusive in biomedicine. The COVID-19 pandemic has highlighted the problems associated with the lack of data sharing. The objective of this article is to draw attention to the problem and possible ways to address it.

STUDY DESIGN AND SETTING

This article examines some of the current open access and data sharing practices at biomedical journals and funders. In the context of COVID-19 the consequences of these practices is also examined.

RESULTS

Despite the best of intentions on the part of funders and journals, COVID-19 biomedical research is not open. Academic institutions need to incentivize and reward data sharing practices as part of researcher assessment. Journals and funders need to implement strong polices to ensure that data sharing becomes a reality. Patients support sharing of their data.

CONCLUSION

Biomedical journals, funders and academic institutions should act to require stronger adherence to data sharing policies.

Status, use and impact of sharing individual participant data from clinical trials: a scoping review

OBJECTIVES

To explore the impact of data-sharing initiatives on the intent to share data, on actual data sharing, on the use of shared data and on research output and impact of shared data.

ELIGIBILITY CRITERIA

All studies investigating data-sharing practices for individual participant data (IPD) from clinical trials.

SOURCES OF EVIDENCE

We searched the Medline database, the Cochrane Library, the Science Citation Index Expanded and the Social Sciences Citation Index via Web of Science, and preprints and proceedings of the International Congress on Peer Review and Scientific Publication. In addition, we inspected major clinical trial data-sharing platforms, contacted major journals/publishers, editorial groups and some funders.

CHARTING METHODS

Two reviewers independently extracted information on methods and results from resources identified using a standardised questionnaire. A map of the extracted data was constructed and accompanied by a narrative summary for each outcome domain.

RESULTS

93 studies identified in the literature search (published between 2001 and 2020, median: 2018) and 5 from additional information sources were included in the scoping review. Most studies were descriptive and focused on early phases of the data-sharing process. While the willingness to share IPD from clinical trials is extremely high, actual data-sharing rates are suboptimal. A survey of journal data suggests poor to moderate enforcement of the policies by publishers. Metrics provided by platforms suggest that a large majority of data remains unrequested. When requested, the purpose of the reuse is more often secondary analyses and meta-analyses, rarely re-analyses. Finally, studies focused on the real impact of data-sharing were rare and used surrogates such as citation metrics.

CONCLUSIONS

There is currently a gap in the evidence base for the impact of IPD sharing, which entails uncertainties in the implementation of current data-sharing policies. High level evidence is needed to assess whether the value of medical research increases with data-sharing practices.

Variation in Calculating and Reporting Antimalarial Efficacy against Plasmodium falciparum in Sub-Saharan Africa: A Systematic Review of Published Reports

Antimalarials, in particular artemisinin-based combination therapies (ACTs), are critical tools in reducing the global burden of malaria, which is concentrated in sub-Saharan Africa. Performing and reporting antimalarial efficacy studies in a transparent and standardized fashion permit comparison of efficacy outcomes across countries and time periods. This systematic review summarizes study compliance with WHO laboratory and reporting guidance pertaining to antimalarial therapeutic efficacy studies and evaluates how well studies from sub-Saharan Africa adhered to these guidelines. We included all published studies (January 2020 or before) performed in sub-Saharan Africa where ACT efficacy for treatment of uncomplicated Plasmodium falciparum infection was reported. The primary outcome was a composite indicator for study methodology consistent with WHO guidelines for statistical analysis of corrected efficacy, defined as an article presenting a Kaplan-Meier survival analysis of corrected efficacy or reporting a per-protocol analysis where new infections were excluded from the numerator and denominator. Of 581 articles screened, we identified 279 for the review. Molecular correction was used in 83% (232/279) to distinguish new infections from recrudescences in subjects experiencing recurrent parasitemia. Only 45% (99/221) of articles with therapeutic efficacy as a primary outcome and performing molecular correction reported corrected efficacy outcomes calculated in a way consistent with WHO recommendations. These results indicate a widespread lack of compliance with WHO-recommended methods of analysis, which may result in biases in how antimalarial effectiveness is being measured and reported from sub-Saharan Africa.