A descriptive analysis of a representative sample of pediatric randomized controlled trials published in 2007

Supporting study registration to reduce research waste

An estimated 82-89% of ecological research and 85% of medical research has limited or no value to the end user because of various inefficiencies. We argue that registration and registered reports can enhance the quality and impact of ecological research. Drawing on evidence from other fields, chiefly medicine, we support our claim that registration can reduce research waste. However, increasing registration rates, quality and impact will be very slow without coordinated effort of funders, publishers and research institutions. We therefore call on them to facilitate the adoption of registration by providing adequate support. We outline several aspects to be considered when designing a registration system that would best serve the field of ecology. To further inform the development of such a system, we call for more research to identify the causes of low registration rates in ecology. We suggest short- and long-term actions to bolster registration and reduce research waste.

Developments in the Design, Conduct, and Reporting of Child Health Trials

To identify priority areas to improve the design, conduct, and reporting of pediatric clinical trials, the international expert network, Standards for Research (StaR) in Child Health, was assembled and published the first 6 Standards in Pediatrics in 2012. After a recent review summarizing the 247 publications by StaR Child Health authors that highlight research practices that add value and reduce research "waste," the current review assesses the progress in key child health trial methods areas: consent and recruitment, containing risk of bias, roles of data monitoring committees, appropriate sample size calculations, outcome selection and measurement, and age groups for pediatric trials. Although meaningful change has occurred within the child health research ecosystem, measurable progress is still disappointingly slow. In this context, we identify and review emerging trends that will advance the agenda of increased clinical usefulness of pediatric trials, including patient and public engagement, Bayesian statistical approaches, adaptive designs, and platform trials. We explore how implementation science approaches could be applied to effect measurable improvements in the design, conducted, and reporting of child health research.

Assessment of Usefulness of Randomized Control Trials in Child Health Research Published in 2007 and 2017

OBJECTIVE

To examine how clinical usefulness in pediatric research with randomized controlled trials (RCTs) has changed over a 10-year period via a research usefulness tool composed of unique clinical usefulness criteria.

STUDY DESIGN

We leveraged a pre-existing sample of child health RCTs published in 2007, used by our team in a previous study. Using the same methods, a research librarian executed a literature search in the Cochrane Central Register of Controlled Trials for the 2017 cohort. We included the first 300 eligible citations from the randomly ordered list for each year, creating two cohorts of 300 publications each, 1 in 2007 and 1 in 2017. Each publication was analyzed and data regarding primary and secondary outcomes, as well as 11 unique criteria of clinical usefulness, were extracted. Each publication was then graded using a tool created by our research team. After quality review, statistical analysis was then performed.

RESULTS

Six hundred pediatric RCT publications were included in this review. The mean score increased from 6.07 in 2007 to 9.20 in 2017 (P < .001). Usefulness factors that saw the largest increase in reporting were context placement, funding statements, and conflict of interest statements, while patient centeredness, value for money, and raw data availability remained infrequently reported.

CONCLUSION

Our results demonstrate that clinical usefulness of pediatric research improved over this 10-year period, but there are still areas that need a great deal of improvement in order to maximize clinical usefulness and reduce research waste.

A Decade of Efforts to Add Value to Child Health Research Practices

OBJECTIVE

To identify practices that add value to improve the design, conduct, and reporting of child health research and reduce research waste.

STUDY DESIGN

In order to categorize the contributions of members of Standards for Research (StaR) in Child Health network, we developed a novel Child Health Improving Research Practices (CHIRP) framework comprised of 5 domains meant to counteract avoidable child health research waste and improve quality: 1) address research questions relevant to children, their families, clinicians, and researchers; 2) apply appropriate research design, conduct and analysis; 3) ensure efficient research oversight and regulation; 4) Provide accessible research protocols and reports; and 5) develop unbiased and usable research reports, including 17 responsible research practice recommendations. All child health research relevant publications by the 48 original StaR standards' authors over the last decade were identified, and main topic areas were categorized using this framework.

RESULTS

A total of 247 publications were included in the final sample: 100 publications (41%) in domain 1 (3 recommendations), 77 publications (31%) in domain 2 (3), 35 publications (14%) in domain 3 (4), 20 publications (8%) in domain 4 (4), and 15 publications (6%) in domain 5 (3). We identified readily implementable "responsible" research practices to counter child health research waste and improve quality, especially in the areas of patients and families' engagement throughout the research process, developing Core Outcome Sets, and addressing ethics and regulatory oversight issues.

CONCLUSION

While most of the practices are readily implementable, increased awareness of methodological issues and wider guideline uptake is needed to improve child health research. The CHIRP Framework can be used to guide responsible research practices that add value to child health research.

Guidelines for reporting pediatric and child health clinical trial protocols and reports: study protocol for SPIRIT-Children and CONSORT-Children

BACKGROUND

Despite the critical importance of clinical trials to provide evidence about the effects of intervention for children and youth, a paucity of published high-quality pediatric clinical trials persists. Sub-optimal reporting of key trial elements necessary to critically appraise and synthesize findings is prevalent. To harmonize and provide guidance for reporting in pediatric controlled clinical trial protocols and reports, reporting guideline extensions to the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) and Consolidated Standards of Reporting Trials (CONSORT) guidelines specific to pediatrics are being developed: SPIRIT-Children (SPIRIT-C) and CONSORT-Children (CONSORT-C).

METHODS

The development of SPIRIT-C/CONSORT-C will be informed by the Enhancing the Quality and Transparency of Health Research Quality (EQUATOR) method for reporting guideline development in the following stages: (1) generation of a preliminary list of candidate items, informed by (a) items developed during initial development efforts and child relevant items from recent published SPIRIT and CONSORT extensions; (b) two systematic reviews and environmental scan of the literature; (c) workshops with young people; (2) an international Delphi study, where a wide range of panelists will vote on the inclusion or exclusion of candidate items on a nine-point Likert scale; (3) a consensus meeting to discuss items that have not reached consensus in the Delphi study and to "lock" the checklist items; (4) pilot testing of items and definitions to ensure that they are understandable, useful, and applicable; and (5) a final project meeting to discuss each item in the context of pilot test results. Key partners, including young people (ages 12-24 years) and family caregivers (e.g., parents) with lived experiences with pediatric clinical trials, and individuals with expertise and involvement in pediatric trials will be involved throughout the project. SPIRIT-C/CONSORT-C will be disseminated through publications, academic conferences, and endorsement by pediatric journals and relevant research networks and organizations.

DISCUSSION

SPIRIT/CONSORT-C may serve as resources to facilitate comprehensive reporting needed to understand pediatric clinical trial protocols and reports, which may improve transparency within pediatric clinical trials and reduce research waste.

TRIAL REGISTRATION

The development of these reporting guidelines is registered with the EQUATOR Network: SPIRIT-Children ( https://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-clinical-trials-protocols/#35 ) and CONSORT-Children ( https://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-clinical-trials/#CHILD ).

Changes to design and analysis elements of research plans during randomised controlled trials in Australia

OBJECTIVES

To investigate the frequency and legitimacy of substantive changes to the research plans of published randomised controlled trials (RCTs) undertaken in Australia.

DESIGN

Comparison of methodology and analysis plans for RCTs specified in protocol documents (full protocols, published protocol articles, statistical analysis plans, Australian New Zealand Clinical Trials Registry [ANZCTR] registration entries) and described in publications of primary results.

SETTING, PARTICIPANTS

181 RCTs registered with the ANZCTR, 1 September 2007 - 31 December 2013, for which primary results had been published.

MAIN OUTCOME MEASURE

Changes made to research plan, both overall and by specific item (primary outcome, analysis set, eligibility criteria, sample size, primary analysis method, and treatment arms included in the primary comparison in multi-arm trials); trial characteristics associated with changes.

RESULTS

Protocol documents were available for 124 of 181 eligible RCTs (69%; 46 publicly available, 78 provided by trial groups on request). Full audit of RCTs with protocols found clear or probable changes in 111 trials (90%), for 101 of which (91%) it was unclear whether changes had been made blinded to treatment outcomes. After seeking clarification from investigators, changes to 78 trials were confirmed (63%), for 61 of which (78%) changes were made blinded to treatment outcomes. Any change was less likely for trials with publicly available protocols than for trials for which we needed to request protocols (odds ratio, 0.22; 95% CI, 0.06-0.77). Limited reviews of trials without protocols identified that changes had been made to 42 of 57 trials (74%).

CONCLUSION

Changes to RCT study plans in Australia are both frequent and usually made appropriately blinded to treatment outcomes. However, the documentation of changes made to RCT protocols should be formalised to improve transparency.

Randomized controlled trials in patients with COVID-19: a systematic review and critical appraisal

OBJECTIVES

This study aimed to describe the prevalence of risks of bias in randomized trials of therapeutic interventions for COVID-19.

METHODS

Systematic review and risk of bias assessment performed by two independent reviewers of a random sample of 40 randomized trials of therapeutic interventions for moderate-severe COVID-19. We used the RoB 2.0 tool to assess the risk of bias, which evaluates bias under five domains as well as an overall assessment of each trial as high or low risk of bias.

RESULTS

Of the 40 included trials, 19 (47%) were at high risk of bias, and this was particularly frequent in trials from low-middle income countries (11/14, 79%). Potential deviations to intended interventions (i.e., control participants accessing experimental treatments) were considered a potential source of bias in some studies (14, 35%), as was the risk due to selective reporting of results (6, 15%). The randomization process was considered at low risk of bias in most studies (34, 95%), as were missing data (36, 90%) and measurement of the outcome (35, 87%).

CONCLUSION

Many randomized trials evaluating COVID-19 interventions are at risk of bias, particularly those conducted in low-middle income countries. Biases are mostly due to deviations from intended interventions and partly due to the selection of reported results. The use of placebo control and publicly available protocol can mitigate many of these risks.

Methodological Quality and Risk of Bias Assessment of Cardiovascular Disease Research: Analysis of Randomized Controlled Trials Published in 2017

Background

All randomized-controlled trials (RCTs) are required to follow high methodological standards. In this study, we aimed to assess the methodological quality of published cardiovascular clinical research trials in a representative sample of RCTs published in 2017.

Methods

Cochrane Central Register of Controlled Trials was used to identify cardiovascular clinical research trials with adult participants published in 2017. Overall, 250 (10%) RCTs were randomly selected from a total of 2,419 studies. Data on general trial characteristics were extracted and the risk of bias (RoB) was determined.

Results

Overall, 86% of RCTs have reported at least one statistically significant result, with the primary outcome significant in 69%, treatment favored in 55%, and adverse events reported in 68%. Less than one-third (29%) of trials were overall low RoB, while the other two-thirds were rated unclear (40%) or with high RoB (31%). Sequence generation, allocation concealment, and selective reporting were the domains most often rated with high RoB. Drug trials were more likely to have low RoB than non-drug trials. Significant differences were found in RoB for the allocation concealment and blinding of participants and personnel between industry-funded and non-industry-funded trials, with industry-funded trials more often rated at low RoB.

Conclusion

Almost two-thirds of RCTs in the field of cardiovascular disease (CVD) research, were at high or unclear RoB, indicating a need for more rigorous trial planning and conduct. Prospective trial registration is a factor predicting a lower risk of bias.

Creating efficiencies in the extraction of data from randomized trials: a prospective evaluation of a machine learning and text mining tool

Background

Machine learning tools that semi-automate data extraction may create efficiencies in systematic review production. We evaluated a machine learning and text mining tool’s ability to (a) automatically extract data elements from randomized trials, and (b) save time compared with manual extraction and verification.

Methods

For 75 randomized trials, we manually extracted and verified data for 21 data elements. We uploaded the randomized trials to an online machine learning and text mining tool, and quantified performance by evaluating its ability to identify the reporting of data elements (reported or not reported), and the relevance of the extracted sentences, fragments, and overall solutions. For each randomized trial, we measured the time to complete manual extraction and verification, and to review and amend the data extracted by the tool. We calculated the median (interquartile range [IQR]) time for manual and semi-automated data extraction, and overall time savings.

Results

The tool identified the reporting (reported or not reported) of data elements with median (IQR) 91% (75% to 99%) accuracy. Among the top five sentences for each data element at least one sentence was relevant in a median (IQR) 88% (83% to 99%) of cases. Among a median (IQR) 90% (86% to 97%) of relevant sentences, pertinent fragments had been highlighted by the tool; exact matches were unreliable (median (IQR) 52% [33% to 73%]). A median 48% of solutions were fully correct, but performance varied greatly across data elements (IQR 21% to 71%). Using ExaCT to assist the first reviewer resulted in a modest time savings compared with manual extraction by a single reviewer (17.9 vs. 21.6 h total extraction time across 75 randomized trials).

Conclusions

Using ExaCT to assist with data extraction resulted in modest gains in efficiency compared with manual extraction. The tool was reliable for identifying the reporting of most data elements. The tool’s ability to identify at least one relevant sentence and highlight pertinent fragments was generally good, but changes to sentence selection and/or highlighting were often required.

Protocol

https://doi.org/10.7939/DVN/RQPJKS

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01354-2.

P value and Bayesian analysis in randomized-controlled trials in child health research published over 10 years, 2007 to 2017: a methodological review protocol

BACKGROUND

There is an unresolved debate about the reliability of the interpretation of P value. Some investigators have suggested that an alternative Bayesian method is preferred in conducting health research. As randomized-controlled trials (RCTs) are important in generating research evidence, we decided to investigate the extent, if any, the inferential statistical framework in published RCTs in child health research have changed over 10 years. We aim to examine the change in P value and Bayesian analysis in RCTs in child health research papers published from 2007 to 2017.

METHODS

We will search the Cochrane Central Register of Controlled Trials (Wiley) to identify relevant citations. We will leverage a pre-existing sample of child health RCTs published in 2007 (n=300) used in our previous study of reporting quality of pediatric RCTs. Using the same strategy and study selection methods, we will identify a comparable random sample of child health RCTs published in 2017 (n=300). Eligible studies will include RCTs in health research among individuals aged 21 years and below. One reviewer will select studies for inclusion and extract the data and another reviewer will verify these. Disagreements will be resolved by a discussion between reviewers or by involving another reviewer. We will perform a descriptive analysis of 2007 and 2017 samples and analyze the results using both the frequentist and Bayesian methods. We will present specific characteristics of the clinical trials from 2007 and 2017 in tabular and graphical forms. We will report the difference in the proportion of P value and Bayesian analysis between 2007 and 2017 to assess the 10-year change. Clustering around P values of significance, if observed, will be reported.

DISCUSSION

This review will present the difference in the proportion of trials that reported on P value and Bayesian analysis between 2007 and 2017 to assess the 10-year change. The implications for future clinical research will be discussed and this research work will be published in a peer-reviewed journal. This review has the potential to help inform the need for a change in the methodological approach from the null hypothesis significance test to Bayesian methods.

SYSTEMATIC REVIEW REGISTRATION

Open Science Framework https://osf.io/aj2df.

Publication Trends of Pediatric and Adult Randomized Controlled Trials in General Medical Journals, 2005-2018: A Citation Analysis

Policy has been developed to promote the conduct of high-quality pediatric randomized controlled trials (RCTs). Whether these strategies have influenced publication trends in high-impact journals is unknown. We aim to evaluate characteristics, citation patterns, and publication trends of pediatric RCTs published in general medical journals (GMJs) compared with adult RCTs over a 13-year period. Studies were identified using Medline, and impact metrics were collected from Web of Science and Scopus. All RCTs published from 2005–2018 in 7 GMJs with the highest impact factors were identified for analysis. A random sample of matched pediatric and adult RCTs were assessed for publication characteristics, academic and non-academic citation. Citations were counted from publication until June 2019. Among 4146 RCTs, 2794 (67.3%) enrolled adults, 591 (14.2%) enrolled children, and 761 RCTs (18.3%) enrolled adult and pediatric patients. Adult RCTs published in GMJs grew by 5.1 publications per year (95% CI: 3.3–6.9), while the number of pediatric RCTs did not show significant change (−0.4 RCTs/year, 95% CI: −1.4–0.6). Adult RCTs were cited more than pediatric RCTs (median(IQR): 29.9 (68.5–462.8) citations/year vs. 13.2 (6.8–24.9) citations/year; p < 0.001); however, social media attention was similar (median(IQR) Altmetric Attention Score: 37 (13.75–133.8) vs. 26 (6.2–107.5); p = 0.25). Despite policies which may facilitate conduct of pediatric RCTs, the publishing gap in high-impact GMJs is widening.

Pediatric Age Groups and Approach to Studies

Pediatric clinical trials are often requested according to specific age ranges. In the past and still today, these ages may correspond to developmental stages, such as newborn, infancy, childhood, and adolescence. Selection of ages for pediatric participation in medication studies should correspond to ages of rapid changes in pharmacokinetics and pharmacodynamics. Age-related changes in several enzymes involved in drug metabolism and glomerular filtration are described as examples of optimal ages for study of specific drugs according to their pathways of disposition.

Special considerations in conducting clinical trials of chronic pain management interventions in children and adolescents and their families

INTRODUCTION

Disabling chronic pain is a common experience for children and adolescents. However, the evidence base for chronic pain interventions for youth is extremely limited, which has hindered the development of evidence-based practice guidelines for most pediatric chronic pain conditions.

OBJECTIVES

To review and provide recommendations on clinical trial design and evaluation in children and adolescents with chronic pain.

METHODS

In this article, we summarize key issues and provide recommendations for addressing them in clinical trials of chronic pain interventions in children and adolescents and their families.

RESULTS

To stimulate high-quality trials of pediatric chronic pain management interventions, attention to key issues including sample characterization, trial design and treatment administration, outcome measurement, and the ethics of intervening with children and adolescents, as opposed to adults with chronic pain, is needed.

CONCLUSION

Future research to develop interventions to reduce or prevent childhood chronic pain is an important priority area, and requires special considerations in implementation and evaluation in clinical trials.

Reporting of data monitoring committees and adverse events in paediatric trials: a descriptive analysis

OBJECTIVES

For 300 paediatric trials, we evaluated the reporting of: a data monitoring committee (DMC); interim analyses, stopping rules and early stopping; and adverse events and harm-related endpoints.

METHODS

For this cross-sectional evaluation, we randomly selected 300 paediatric trials published in 2012 from the Cochrane Central Register of Controlled Trials. We collected data on the reporting of a DMC; interim analyses, stopping rules and early stopping; and adverse events and harm-related endpoints. We reported the findings descriptively and stratified by trial characteristics.

RESULTS

Eighty-five (28%) of the trials investigated drugs, and 18% (n=55/300) reported a DMC. The reporting of a DMC was more common among multicentre than single centre trials (n=41/132, 31% vs n=14/139, 10%, p<0.001) and industry-sponsored trials compared with those sponsored by other sources (n=16/50, 32% vs n=39/250, 16%, p=0.009). Trials that reported a DMC enrolled more participants than those that did not (median [range]): 224 (10-60480) vs 91 (10-9528) (p<0.001). Only 25% of these trials reported interim analyses, and 42% reported stopping rules. Less than half (n=143/300, 48%) of trials reported on adverse events, and 72% (n=215/300) reported on harm-related endpoints. Trials that reported a DMC compared with those that did not were more likely to report adverse events (n=43/55, 78% vs 100/245, 41%, p<0.001) and harm-related endpoints (n=52/55, 95% vs. 163/245, 67%, p<0.001). Only 32% of drug trials reported a DMC; 18% and 19% did not report on adverse events or harm-related endpoints, respectively.

CONCLUSIONS

The reporting of a DMC was infrequent, even among drug trials. Few trials reported stopping rules or interim analyses. Reporting of adverse events and harm-related endpoints was suboptimal.

Are Neonatal Trials Better Conducted and Reported over the Last 6 Decades? An Analysis on Their Risk-of-Bias Status in Cochrane Reviews

BACKGROUND

The introduction of Neonatology as a subspecialty in 1960 has stimulated an enormous amount of neonatal research. A large proportion of neonatal randomized-controlled trials (RCTs) have been included in the Cochrane reviews, within which methodological quality or risk-of-bias (ROB) assessment is an integral feature.

OBJECTIVES

We described the ROB profile of neonatal RCTs published since the 1950s.

METHODS

We analyzed individual studies within the Cochrane Neonatal reviews published up to December 2016. We extracted the reviewers' judgments on the ROB domains including random sequence generation, allocation concealment, blinding, incomplete outcome data, and selective reporting. We evaluated blinding of personnel in trials in which blinding was considered feasible.

RESULTS

We assessed 1980 RCTs published between 1952 and 2016 from 294 Cochrane Neonatal systematic reviews, with full ROB assessments performed in 848 trials (42.8%). Among the ROB domains, the highest proportion of trials (73%) were judged as satisfactory ("low risk") in handling incomplete outcome data, while fewest trials achieved blinding of outcome assessor (38.4%). In the last 6 decades, a progressive increase has been observed in the proportion of trials that were rated as low risk in random sequence generation, allocation concealment, and selective reporting. However, blinding was achieved in less than half of the trials with no clear improvement across decades (23-44% since the 1980s).

CONCLUSIONS

Despite steady improvement in the overall quality of neonatal RCTs over the last 6 decades, blinding remained unsatisfactory in the majority of the trials.

Registration of published randomized trials: a systematic review and meta-analysis

BACKGROUND

Prospective trial registration is a powerful tool to prevent reporting bias. We aimed to determine the extent to which published randomized controlled trials (RCTs) were registered and registered prospectively.

METHODS

We searched MEDLINE and EMBASE from January 2005 to October 2017; we also screened all articles cited by or citing included and excluded studies, and the reference lists of related reviews. We included studies that examined published RCTs and evaluated their registration status, regardless of medical specialty or language. We excluded studies that assessed RCT registration status only through mention of registration in the published RCT, without searching registries or contacting the trial investigators. Two independent reviewers blinded to the other's work performed the selection. Following PRISMA guidelines, two investigators independently extracted data, with discrepancies resolved by consensus. We calculated pooled proportions and 95% confidence intervals using random-effects models.

RESULTS

We analyzed 40 studies examining 8773 RCTs across a wide range of clinical specialties. The pooled proportion of registered RCTs was 53% (95% confidence interval 44% to 58%), with considerable between-study heterogeneity. A subset of 24 studies reported data on prospective registration across 5529 RCTs. The pooled proportion of prospectively registered RCTs was 20% (95% confidence interval 15% to 25%). Subgroup analyses showed that registration was higher for industry-supported and larger RCTs. A meta-regression analysis across 19 studies (5144 RCTs) showed that the proportion of registered trials significantly increased over time, with a mean proportion increase of 27%, from 25 to 52%, between 2005 and 2015.

CONCLUSIONS

The prevalence of trial registration has increased over time, but only one in five published RCTs is prospectively registered, undermining the validity and integrity of biomedical research.

Reporting of interventions and "standard of care" control arms in pediatric clinical trials: a quantitative analysis

BACKGROUND

In pediatric medicine, the usual treatment received by children ("standard of care") varies across centers. Evaluations of new treatments often compare to the existing "standard of care" to determine if a treatment is more effective, has a better safety profile, or costs less. The objective of our study was to evaluate intervention and "standard of care" control arms reported in published pediatric clinical trials.

METHODS

Pediatric clinical trials, published in 2014, reporting the use of a "standard of care" control arm were included. Duplicate assessment of reporting completeness was done using the 12-item TIDieR (Template for Intervention Description and Replication) checklist for both the "standard of care" control arms and intervention arms within the same published study.

RESULTS

Following screening, 214 pediatric trials in diverse therapeutic areas were included. Several different terms were used to describe "standard of care." There was a significant difference between the mean reported TIDieR checklist items of "standard of care" control arms (5.81 (SD 2.13) and intervention arms (8.45 (SD 1.39, p < 0.0001).

CONCLUSIONS

Reporting of intervention and "standard of care" control arms in pediatric clinical trials should be improved as current "standard of care" reporting deficiencies limit reproducibility of research and may ultimately contribute to research waste.

Critical components for designing and implementing randomized controlled trials

Randomized controlled trials (RCTs) are considered the first level of evidence to assess the efficacy of novel interventions/therapies. Proper design and implementation of an RCT can result in convincing causal inferences. RCTs often represent the gold standard for clinical trials when appropriately designed, conducted and reported. However, there are limitations in implementation of RCTs, including sufficiency of randomized allocation (especial for allocation concealment), implementing standard intervention, maintaining follow-up and statement of conflicting interests. Therefore, the basic principles of RCTs are outlined here so that pediatric investigators can further understand what is the best evidence based on RCTs. More importantly, the quality of pediatric RCTs may be improved by following challenges in pediatric clinical trials outlined here.

An Adaptation of the Original Fresno Test to Measure Evidence-Based Practice Competence in Pediatric Bedside Nurses

PURPOSE

Instruments used to assess evidence-based practice (EBP) competence in nurses have been subjective, unreliable, or invalid. The Fresno test was identified as the only instrument to measure all the steps of EBP with supportive reliability and validity data. However, the items and psychometric properties of the original Fresno test are only relevant to measure EBP with medical residents. Therefore, the purpose of this paper is to describe the development of the adapted Fresno test for pediatric nurses, and provide preliminary validity and reliability data for its use with Bachelor of Science in Nursing-prepared pediatric bedside nurses.

METHODS

General adaptations were made to the original instrument's case studies, item content, wording, and format to meet the needs of a pediatric nursing sample. The scoring rubric was also modified to complement changes made to the instrument. Content and face validity, and intrarater reliability of the adapted Fresno test were assessed during a mixed-methods pilot study conducted from October to December 2013 with 29 Bachelor of Science in Nursing-prepared pediatric nurses.

RESULTS

Validity data provided evidence for good content and face validity. Intrarater reliability estimates were high.

LINKING EVIDENCE TO ACTION

The adapted Fresno test presented here appears to be a valid and reliable assessment of EBP competence in Bachelor of Science in Nursing-prepared pediatric nurses. However, further testing of this instrument is warranted using a larger sample of pediatric nurses in diverse settings. This instrument can be a starting point for evaluating the impact of EBP competence on patient outcomes.

Randomized controlled trials in children's heart surgery in the 21st century: a systematic review

OBJECTIVES

Randomized controlled trials are the gold standard for evaluating health care interventions, yet are uncommon in children's heart surgery. We conducted a systematic review of clinical trials in paediatric cardiac surgery to evaluate the scope and quality of the current international literature.

METHODS

We searched MEDLINE, CENTRAL and LILACS, and manually screened retrieved references and systematic reviews to identify all randomized controlled trials reporting the effect of any intervention on the conduct or outcomes of heart surgery in children published in any language since January 2000; secondary publications and those reporting inseparable adult data were excluded. Two reviewers independently screened studies for eligibility and extracted data; the Cochrane Risk of Bias tool was used to assess for potential biases.

RESULTS

We identified 333 trials from 34 countries randomizing 23 902 children. Most were early phase (313, 94.0%), recruiting few patients (median 45, interquartile range 28-82), and only 11 (3.3%) directly evaluated a surgical intervention. One hundred and nine (32.7%) trials calculated a sample size, 52 (15.6%) reported a CONSORT diagram, 51 (15.3%) were publicly registered and 25 (7.5%) had a Data Monitoring Committee. The overall risk of bias was low in 22 (6.6%), high in 69 (20.7%) and unclear in 242 (72.7%).

CONCLUSIONS

The recent literature in children's heart surgery contains few late-phase clinical trials. Most trials did not conform to the accepted standards of reporting, and the overall risk of bias was low in few studies. There is a need for high-quality, multicentre clinical trials to provide a robust evidence base for contemporary paediatric cardiac surgical practice.

Promoting public access to clinical trial protocols: challenges and recommendations

BACKGROUND

Recognizing the value of promoting public access to clinical trial protocols, Trials pioneered the way for their publication over a decade ago. However, despite major advances in the public accessibility of information about trial methods and results, protocol sharing remains relatively rare.

MAIN BODY

Protocol sharing facilitates the critical appraisal of clinical trials and helps to identify and deter the selective reporting of outcomes and analyses. Challenges to the routine availability of high quality trial protocols include the gaps in incentives and adherence mechanisms, limited venues for sharing the original and final protocol versions, and the need for mechanisms to ensure transparent and complete protocol content.

CONCLUSIONS

We propose recommendations for addressing key challenges to protocol sharing in order to promote routine public access to protocols for the benefit of patients and other users of evidence from clinical trials.

EAP viewpoint on unpublished data from paediatric clinical trials

European children and paediatricians rely heavily on the fair, complete and timely publication of data obtained from paediatric randomised controlled trials (RCTs). Selective publication and reporting of paediatric RCTs is common practice. Industry-sponsored trials are more likely to remain unpublished, and take longer to get published compared with trials sponsored by others. However, also academic sponsors contribute to inefficiencies in publishing clinical data. Publication bias violates the ethical obligation that investigators have towards study participants, leads to considerable inefficiencies in research and a waste of financial and human resources, and has the potential to distort evidence for treatment approaches.

CONCLUSION

The European Academy of Paediatrics (EAP) therefore actively supports initiatives that increase the public dissemination of paediatric clinical trial data. The EAP will raise awareness about the guidelines for Good Publication Practice among European paediatricians and subspecialty societies.

The Conduct and Reporting of Child Health Research: An Analysis of Randomized Controlled Trials Published in 2012 and Evaluation of Change over 5 Years

OBJECTIVES

For child health randomized controlled trials (RCTs) published in 2012, we aimed to describe design and reporting characteristics and evaluate changes since 2007; assess the association between trial design and registration and risk of bias (RoB); and assess the association between RoB and effect size.

STUDY DESIGN

For 300 RCTs, we extracted design and reporting characteristics and assessed RoB. We assessed 5-year changes in design and reporting (based on 300 RCTs we had previously analyzed) using the Fisher exact test. We tested for associations between design and reporting characteristics and overall RoB and registration using the Fisher exact, Cochran-Armitage, Kruskal-Wallis, and Jonckheere-Terpstra tests. We pooled effect sizes and tested for differences by RoB using the χ² test for subgroups in meta-analysis.

RESULTS

The 2012 and 2007 RCTs differed with respect to many design and reporting characteristics. From 2007 to 2012, RoB did not change for random sequence generation and improved for allocation concealment (P < .001). Fewer 2012 RCTs were rated high overall RoB and more were rated unclear (P = .03). Only 7.3% of 2012 RCTs were rated low overall RoB. Trial registration doubled from 2007 to 2012 (23% to 46%) (P < .001) and was associated with lower RoB (P = .009). Effect size did not differ by RoB (P = .43) CONCLUSIONS: Random sequence generation and allocation concealment were not often reported, and selective reporting was prevalent. Measures to increase trialists' awareness and application of existing reporting guidance, and the prospective registration of RCTs is needed to improve the trustworthiness of findings from this field.

Consent and recruitment: the reporting of paediatric trials published in 2012

OBJECTIVES

We evaluated 300 paediatric trials to determine: the consent and recruitment strategies used, who trial information was targeted to, how incentives were used and if they achieved their recruitment targets.

METHODS

For this cross-sectional evaluation, we searched the Cochrane Central Register of Controlled Trials for paediatric trials published in 2012 and randomly selected 300 that reported on outcomes for participants aged ≤21 years. We collected data on consent and recruitment procedures for each trial and undertook descriptive analyses in SPSS statistics V.23.

RESULTS

All but one trial (99.7%) used a standard recruitment strategy. Most (92%) trials reported that consent was obtained but only 13% reported who obtained consent. Two-thirds (65%) of trials included school-aged participants, and of these 68% reported obtaining assent. Half (50%) of the trials reported who the trial information was targeted to. Most trials (75%) of school-aged participants targeted information towards children or children and their parents. Fourteen per cent of trials reported using incentives, half (50%) of which were in the form of compensation. Only 48% of trials reported sufficient data to determine if their recruitment targets were achieved. Of these, 70% achieved their targets.

CONCLUSIONS

Notable reporting shortcomings included: how families were recruited into the trial, who obtained consent and/or assent and how, who trial information was directed to, whether incentives were used and sufficient data to determine if the recruitment target was achieved. Forthcoming paediatric-specific reporting standards may improve reporting in this priority area. Our data provide a baseline for ongoing monitoring of the state of the research.

Randomized clinical trials in dentistry: Risks of bias, risks of random errors, reporting quality, and methodologic quality over the years 1955-2013

OBJECTIVES

To examine the risks of bias, risks of random errors, reporting quality, and methodological quality of randomized clinical trials of oral health interventions and the development of these aspects over time.

METHODS

We included 540 randomized clinical trials from 64 selected systematic reviews. We extracted, in duplicate, details from each of the selected randomized clinical trials with respect to publication and trial characteristics, reporting and methodologic characteristics, and Cochrane risk of bias domains. We analyzed data using logistic regression and Chi-square statistics.

RESULTS

Sequence generation was assessed to be inadequate (at unclear or high risk of bias) in 68% (n = 367) of the trials, while allocation concealment was inadequate in the majority of trials (n = 464; 85.9%). Blinding of participants and blinding of the outcome assessment were judged to be inadequate in 28.5% (n = 154) and 40.5% (n = 219) of the trials, respectively. A sample size calculation before the initiation of the study was not performed/reported in 79.1% (n = 427) of the trials, while the sample size was assessed as adequate in only 17.6% (n = 95) of the trials. Two thirds of the trials were not described as double blinded (n = 358; 66.3%), while the method of blinding was appropriate in 53% (n = 286) of the trials. We identified a significant decrease over time (1955-2013) in the proportion of trials assessed as having inadequately addressed methodological quality items (P < 0.05) in 30 out of the 40 quality criteria, or as being inadequate (at high or unclear risk of bias) in five domains of the Cochrane risk of bias tool: sequence generation, allocation concealment, incomplete outcome data, other sources of bias, and overall risk of bias.

CONCLUSIONS

The risks of bias, risks of random errors, reporting quality, and methodological quality of randomized clinical trials of oral health interventions have improved over time; however, further efforts that contribute to the development of more stringent methodology and detailed reporting of trials are still needed.

A systematic review of pediatric clinical trials of high dose vitamin D

Background. Due to inadequate UV exposure, intake of small quantities of vitamin D is recommended to prevent musculoskeletal disease. Both basic science and observational literature strongly suggest that higher doses may benefit specific populations and have non-musculoskeletal roles. Evaluating the evidence surrounding high dose supplementation can be challenging given a relatively large and growing body of clinical trial evidence spanning time, geography, populations and dosing regimens. Study objectives were to identify and summarize the clinical trial literature, recognize areas with high quality evidence, and develop a resource database that makes the literature more immediately accessible to end users. Methods. Medline (1946 to January 2015), Embase (1974 to January 2015), and Cochrane databases (January 2015), were searched for trials. All pediatric (0-18 years) trials administering doses higher than 400 IU (<1 year) or 600 IU (≥1 year) were included. Data was extracted independently by two of the authors. An online searchable database of trials was developed containing relevant extracted information (http://www.cheori.org/en/pedvitaminddatabaseOverview). Sensitivity and utility were assessed by comparing the trials in the database with those from systematic reviews of vitamin D supplementation including children. Results. A total of 2,579 candidate papers were identified, yielding 169 trials having one or more arms meeting eligibility criteria. The publication rate has increased significantly from 1 per year (1970-1979) to 14 per year (2010-2015). Although 84% of the total trials focused on healthy children or known high risk populations (e.g., renal, prematurity), this proportion has declined in recent years due to the rise in trials evaluating populations and outcomes not directly related to the musculoskeletal actions of vitamin D (27% in 2010s). Beyond healthy children, the only pediatric populations with more than 50 participants from low risk of bias trials evaluating a clinically relevant outcome were prematurity and respiratory illness. Finally, we created and validated the online searchable database using 13 recent systematic reviews. Of the 38 high dose trials identified by the systematic review, 36 (94.7%) could be found within the database. When compared with the search strategy reported in each systematic review, use of the database reduced the number of full papers to assess for eligibility by 85.2% (±13.4%). Conclusion. The pediatric vitamin D field is highly active, with a significant increase in trials evaluating non-classical diseases and outcomes. Despite the large overall number there are few high quality trials of sufficient size to provide answers on clinical efficacy of high-dose vitamin D. An open access online searchable data should assist end users in the rapid and comprehensive identification and evaluation of trials relevant to their population or question of interest.

HELPinKids&Adults Knowledge Synthesis of the Management of Vaccination Pain and High Levels of Needle Fear: Limitations of the Evidence and Recommendations for Future Research

The HELPinKids&Adults knowledge synthesis for the management of vaccination-related pain and high levels of needle fear updated and expanded upon the 2010 HELPinKIDS knowledge synthesis and clinical practice guideline for pain mitigation during vaccine injections in childhood. Interventions for vaccine pain management in adults and treatment of individuals with high levels of needle fear, phobias, or both were included, thereby broadening the reach of this work. The present paper outlines the overarching limitations of this diverse evidence base and provides recommendations for future research. Consistent with the framing of clinical questions in the systematic reviews, the Participants, Intervention, Comparison, Outcome, Study design (PICOAS) framework was used to organize these predominant issues and research directions. The major limitations we identified across systematic reviews were an overall dearth of trials on vaccination, lack of methodological rigor, failure to incorporate important outcomes, poor study reporting, and various sources of heterogeneity. Future research directions in terms of conducting additional trials in the vaccination context, improving methodological quality and rigor, assessment of global acceptability and feasibility of interventions, and inclusion of outcomes that stakeholders consider to be important (eg, compliance) are recommended. Given concerns about pain and fear are known contributors to vaccine hesitancy, improving and expanding this evidence base will be integral to broader efforts to improve vaccine compliance and public health worldwide.

Comparison of Drug Utilization Patterns in Observational Data: Antiepileptic Drugs in Pediatric Patients

PURPOSE

Physicians require information on the comparative benefits and harms of medications for optimal treatment decisions. However, this type of data is limited, especially for pediatric patients.

OBJECTIVE

Our aim was to use observational data to measure and compare medication utilization patterns in a pediatric patient population.

METHODS

Using pharmacy claims data from a large, national-scale insurance program in the USA, we identified all patients with a diagnosis of epilepsy treated with a first-generation antiepileptic drug (carbamazepine, ethosuximide, phenobarbital, phenytoin, or valproate) or a second-generation antiepileptic drug [carbamazepine extended release (XR), gabapentin, lamotrigine, levetiracetam, oxcarbazepine, tiagabine, topiramate, valproate XR, or zonisamide]. Treatment periods were defined on the basis of prescription fill dates and medication days supplied. Medication use was measured for individual antiepileptic drugs and for first-generation and second-generation drugs as groups.

RESULTS

There were 2527 patients (54 %) who initiated therapy with first-generation antiepileptics and 2139 patients (46 %) who initiated therapy with second-generation antiepileptics. First- and second-generation drugs had the same 1-year retention rates [26 % (95 % confidence interval (CI) 24-28) and 26 % (95 % CI 25-28), respectively], and 26 % of patients (95 % CI 25-28) and 29 % of patients (95 % CI 27-31) who started on a first- or second-generation antiepileptic medication, respectively, resumed treatment with the initial drug after discontinuation. Overall, 73 % of patients (95 % CI 71-74) were treated with only one antiepileptic drug, with similar rates for patients started on first- and second-generation drugs [71 % (95 % CI 69-73) versus 74 % (95 % CI 72-76)].

CONCLUSION

Comparing drug utilization patterns in a pediatric population using observational data, we found similar rates of retention and therapeutic changes. These findings are consistent with the available comparative data and demonstrate an approach that could be extended to other drug classes and conditions in pediatric populations to examine drug effectiveness.

Recommendations and evidence for reporting items in pediatric clinical trial protocols and reports: two systematic reviews

BACKGROUND

Complete and transparent reporting of clinical trial protocols and reports ensures that these documents are useful to all stakeholders, that bias is minimized, and that the research is not wasted. However, current studies repeatedly conclude that pediatric trial protocols and reports are not appropriately reported. Guidelines like SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) may improve reporting, but do not offer guidance on issues unique to pediatric trials. This paper reports two systematic reviews conducted to build the evidence base for the development of pediatric reporting guideline extensions: 1) SPIRIT-Children (SPIRIT-C) for pediatric trial protocols, and 2) CONSORT-Children (CONSORT-C) for pediatric trial reports.

METHOD

MEDLINE, the Cochrane Methodology Register, and reference lists of included studies were searched. Publications of any type were eligible if they included explicit recommendations or empirical evidence for the reporting of potential items in a pediatric protocol (SPIRIT-C systematic review) or trial report (CONSORT-C systematic review). Study characteristics, recommendations and evidence for pediatric extension items were extracted. Recurrent themes in the recommendations and evidence were identified and synthesized. All steps were conducted by two reviewers.

RESULTS

For the SPIRIT-C and CONSORT-C systematic reviews 366 and 429 publications were included, respectively. Recommendations were identified for 48 of 50 original reporting items and sub-items from SPIRIT, 15 of 20 potential SPIRIT-C reporting items, all 37 original CONSORT items and sub-items, and 16 of 22 potential CONSORT-C reporting items. The following overarching themes of evidence to support or refute the utility of reporting items were identified: transparency; reproducibility; interpretability; usefulness; internal validity; external validity; reporting bias; publication bias; accountability; scientific soundness; and research ethics.

CONCLUSION

These systematic reviews are the first to systematically gather evidence and recommendations for the reporting of specific items in pediatric protocols and trials. They provide useful and translatable evidence on which to build pediatric extensions to the SPIRIT and CONSORT reporting guidelines. The resulting SPIRIT-C and CONSORT-C will provide guidance to the authors of pediatric protocols and reports, respectively, helping to alleviate concerns of inappropriate and inconsistent reporting, and reduce research waste.

StaR Child Health: developing evidence-based guidance for the design, conduct and reporting of paediatric trials

There has been a huge upsurge in clinical research in children in the last decade, stimulated in England by dedicated research infrastructure and support through the National Institute for Health Research. This infrastructure offering research design, expert review, trial management, research nurse, data support and dedicated facilities enables paediatricians to conduct more and better research. The challenge is how to design and conduct trials that will make a real difference to children's health. Standards for Research (StaR) in Child Health was founded in 2009 to address the paucity and shortcomings of paediatric clinical trials. This global initiative involves methodologists, clinicians, patient advocacy groups and policy makers dedicated to developing practical, evidence-based standards for enhancing the reliability and relevance of paediatric clinical research. In this overview, we highlight the contribution of StaR to this agenda, describe the international context, and suggest how StaR's future plans could be integrated with new and existing support for research.

Systematic review and meta-analysis of distraction and hypnosis for needle-related pain and distress in children and adolescents

OBJECTIVE

To systematically review the evidence (and quality) for distraction and hypnosis for needle-related pain and distress in children and adolescents. To explore the effects of distraction characteristics (e.g., adult involvement, type of distracter), child age, and study risk of bias on treatment efficacy. 

METHODS

26 distraction and 7 hypnosis trials were included and self-report, observer-report, and behavioral pain intensity and distress examined. Distraction studies were coded for 4 intervention characteristics, and all studies coded for child age and study risk of bias. 

RESULTS

Findings showed strong support for distraction and hypnosis for reducing pain and distress from needle procedures. The quality of available evidence was low, however. Characteristics of distraction interventions, child age, and study risk of bias showed some influence on treatment efficacy.

CONCLUSIONS

Distraction and hypnosis are efficacious in reducing needle-related pain and distress in children. The quality of trials in this area needs to be improved.

Systematic review and meta-analysis of distraction and hypnosis for needle-related pain and distress in children and adolescents

OBJECTIVE

To systematically review the evidence (and quality) for distraction and hypnosis for needle-related pain and distress in children and adolescents. To explore the effects of distraction characteristics (e.g., adult involvement, type of distracter), child age, and study risk of bias on treatment efficacy. 

METHODS

26 distraction and 7 hypnosis trials were included and self-report, observer-report, and behavioral pain intensity and distress examined. Distraction studies were coded for 4 intervention characteristics, and all studies coded for child age and study risk of bias. 

RESULTS

Findings showed strong support for distraction and hypnosis for reducing pain and distress from needle procedures. The quality of available evidence was low, however. Characteristics of distraction interventions, child age, and study risk of bias showed some influence on treatment efficacy.

CONCLUSIONS

Distraction and hypnosis are efficacious in reducing needle-related pain and distress in children. The quality of trials in this area needs to be improved.

Quantifying bias in randomized controlled trials in child health: a meta-epidemiological study

Objective

To quantify bias related to specific methodological characteristics in child-relevant randomized controlled trials (RCTs).

Design

Meta-epidemiological study.

Data Sources

We identified systematic reviews containing a meta-analysis with 10–40 RCTs that were relevant to child health in the Cochrane Database of Systematic Reviews.

Data Extraction

Two reviewers independently assessed RCTs using items in the Cochrane Risk of Bias tool and other study factors. We used meta-epidemiological methods to assess for differences in effect estimates between studies classified as high/unclear vs. low risk of bias.

Results

We included 287 RCTs from 17 meta-analyses. The proportion of studies at high/unclear risk of bias was: 79% sequence generation, 83% allocation concealment, 67% blinding of participants, 47% blinding of outcome assessment, 49% incomplete outcome data, 32% selective outcome reporting, 44% other sources of bias, 97% overall risk of bias, 56% funding, 35% baseline imbalance, 13% blocked randomization in unblinded trials, and 1% early stopping for benefit. We found no significant differences in effect estimates for studies that were high/unclear vs. low risk of bias for any of the risk of bias domains, overall risk of bias, or other study factors.

Conclusions

We found no differences in effect estimates between studies based on risk of bias. A potential explanation is the number of trials included, in particular the small number of studies with low risk of bias. Until further evidence is available, reviewers should not exclude RCTs from systematic reviews and meta-analyses based solely on risk of bias particularly in the area of child health.

Randomized controlled trials in pediatric critical care: a scoping review

Introduction

Evidence from randomized controlled trials (RCTs) is required to guide treatment of critically ill children, but the number of RCTs available is limited and the publications are often difficult to find. The objectives of this review were to systematically identify RCTs in pediatric critical care and describe their methods and reporting.

Methods

We searched MEDLINE, EMBASE, LILACS and CENTRAL (from inception to April 16, 2013) and reference lists of included RCTs and relevant systematic reviews. We included published RCTs administering any intervention to children in a pediatric ICU. We excluded trials conducted in neonatal ICUs, those enrolling exclusively preterm infants, and individual patient crossover trials. Pairs of reviewers independently screened studies for eligibility, assessed risk of bias, and abstracted data. Discrepancies were resolved by consensus.

Results

We included 248 RCTs: 45 (18%) were multicentered and 14 (6%) were multinational. Trials most frequently enrolled both medical and surgical patients (43%) but postoperative cardiac surgery was the single largest population studied (19%). The most frequently evaluated types of intervention were medications (63%), devices (11%) and nutrition (8%). Laboratory or physiological measurements were the most frequent type of primary outcomes (18%). Half of these trials (50%) reported blinding. Of the 107 (43%) trials that reported an a priori sample size, 34 (32%) were stopped early. The median number of children randomized per trial was 49 and ranged from 6 to 4,947. The frequency of RCT publications increased at a mean rate of 0.7 RCTs per year (P<0.001) from 1 to 20 trials per year.

Conclusions

This scoping review identified the available RCTs in pediatric critical care and made them accessible to clinicians and researchers (http://epicc.mcmaster.ca). Most focused on medications and intermediate or surrogate outcomes, were single-centered and were conducted in North America and Western Europe. The results of this review underscore the need for trials with rigorous methodology, appropriate outcome measures, and improved quality of reporting to ensure that high quality evidence exists to support clinical decision-making in this vulnerable population.

10-year trend in quantity and quality of pediatric randomized controlled trials published in mainland China: 2002-2011

Background

Quality assessment of pediatric randomized controlled trials (RCTs) in China is limited. The aim of this study was to evaluate the quantitative trends and quality indicators of RCTs published in mainland China over a recent 10-year period.

Methods

We individually searched all 17 available pediatric journals published in China from January 1, 2002 to December 30, 2011 to identify RCTs of drug treatment in participants under the age of 18 years. The quality was evaluated according to the Cochrane quality assessment protocol.

Results

Of 1287 journal issues containing 44398 articles, a total of 2.4% (1077/44398) articles were included in the analysis. The proportion of RCTs increased from 0.28% in 2002 to 0.32% in 2011. Individual sample sizes ranged from 10 to 905 participants (median 81 participants); 2.3% of the RCTs were multiple center trials; 63.9% evaluated Western medicine, 32.5% evaluated traditional Chinese medicine; 15% used an adequate method of random sequence generation; and 10.4% used a quasi-random method for randomization. Only 1% of the RCTs reported adequate allocation concealment and 0.6% reported the method of blinding. The follow-up period was from 7 days to 96 months, with a median of 7.5 months. There was incomplete outcome data reported in 8.3%, of which 4.5% (4/89) used intention-to-treat analysis. Only 0.4% of the included trials used adequate random sequence allocation, concealment and blinding. The articles published from 2007 to 2011 revealed an improvement in the randomization method compared with articles published from 2002 to 2006 (from 2.7% to 23.6%, p = 0.000).

Conclusions

In mainland China, the quantity of RCTs did not increase in the pediatric population, and the general quality was relatively poor. Quality improvements were suboptimal in the later 5 years.

Education in health research methodology: use of a wiki for knowledge translation

Introduction

A research-practice gap exists between what is known about conducting methodologically rigorous randomized controlled trials (RCTs) and what is done. Evidence consistently shows that pediatric RCTs are susceptible to high risk of bias; therefore novel methods of influencing the design and conduct of trials are required. The objective of this study was to develop and pilot test a wiki designed to educate pediatric trialists and trainees in the principles involved in minimizing risk of bias in RCTs. The focus was on preliminary usability testing of the wiki.

Methods

The wiki was developed through adaptation of existing knowledge translation strategies and through tailoring the site to the identified needs of the end-users. The wiki was evaluated for usability and user preferences regarding the content and formatting. Semi-structured interviews were conducted with 15 trialists and systematic reviewers, representing varying levels of experience with risk of bias or the conduct of trials. Data were analyzed using content analysis.

Results

Participants found the wiki to be well organized, easy to use, and straightforward to navigate. Suggestions for improvement tended to focus on clarification of the text or on esthetics, rather than on the content or format. Participants liked the additional features of the site that were supplementary to the text, such as the interactive examples, and the components that focused on practical applications, adding relevance to the theory presented. While the site could be used by both trialists and systematic reviewers, the lack of a clearly defined target audience caused some confusion among participants.

Conclusions

Participants were supportive of using a wiki as a novel educational tool. The results of this pilot test will be used to refine the risk of bias wiki, which holds promise as a knowledge translation intervention for education in medical research methodology.

Do health care institutions value research? A mixed methods study of barriers and facilitators to methodological rigor in pediatric randomized trials

BACKGROUND

Pediatric randomized controlled trials (RCTs) are susceptible to a high risk of bias. We examined the barriers and facilitators that pediatric trialists face in the design and conduct of unbiased trials.

METHODS

We used a mixed methods design, with semi-structured interviews building upon the results of a quantitative survey. We surveyed Canadian (n=253) and international (n=600) pediatric trialists regarding their knowledge and awareness of bias and their perceived barriers and facilitators in conducting clinical trials. We then interviewed 13 participants from different subspecialties and geographic locations to gain a more detailed description of how their experiences and attitudes towards research interacted with trial design and conduct.

RESULTS

The survey response rate was 23.0% (186/807). 68.1% of respondents agreed that bias is a problem in pediatric RCTs and 72.0% felt that there is sufficient evidence to support changing some aspects of how trials are conducted. Knowledge related to bias was variable, with inconsistent awareness of study design features that may introduce bias into a study. Interview participants highlighted a lack of formal training in research methods, a negative research culture, and the pragmatics of trial conduct as barriers. Facilitators included contact with knowledgeable and supportive colleagues and infrastructure for research.

CONCLUSIONS

A lack of awareness of bias and negative attitudes towards research present significant barriers in terms of conducting methodologically rigorous pediatric RCTs. Knowledge translation efforts must focus on these issues to ensure the relevance and validity of trial results.

Pediatric versus adult drug trials for conditions with high pediatric disease burden

BACKGROUND AND OBJECTIVE

Optimal treatment decisions in children require sufficient evidence on the safety and efficacy of pharmaceuticals in pediatric patients. However, there is concern that not enough trials are conducted in children and that pediatric trials differ from those performed in adults. Our objective was to measure the prevalence of pediatric studies among clinical drug trials and compare trial characteristics and quality indicators between pediatric and adult drug trials.

METHODS

For conditions representing a high burden of pediatric disease, we identified all drug trials registered in ClinicalTrials.gov with start dates between 2006 and 2011 and tracked the resulting publications. We measured the proportion of pediatric trials and subjects for each condition and compared pediatric and adult trial characteristics and quality indicators.

RESULTS

For the conditions selected, 59.9% of the disease burden was attributable to children, but only 12.0% (292/2440) of trials were pediatric (P < .001). Among pediatric trials, 58.6% were conducted without industry funding compared with 35.0% of adult trials (P < .001). Fewer pediatric compared with adult randomized trials examined safety outcomes (10.1% vs 16.9%, P = .008). Pediatric randomized trials were slightly more likely to be appropriately registered before study start (46.9% vs 39.3%, P = .04) and had a modestly higher probability of publication in the examined time frame (32.8% vs 23.2%, P = .04).

CONCLUSIONS

There is substantial discrepancy between pediatric burden of disease and the amount of clinical trial research devoted to pediatric populations. This may be related in part to trial funding, with pediatric trials relying primarily on government and nonprofit organizations.

Empirical evaluation of age groups and age-subgroup analyses in pediatric randomized trials and pediatric meta-analyses

BACKGROUND

An important step toward improvement of the conduct of pediatric clinical research is the standardization of the ages of children to be included in pediatric trials and the optimal age-subgroups to be analyzed.

METHODS

We set out to evaluate empirically the age ranges of children, and age-subgroup analyses thereof, reported in recent pediatric randomized clinical trials (RCTs) and meta-analyses. First, we screened 24 RCTs published in Pediatrics during the first 6 months of 2011; second, we screened 188 pediatric RCTs published in 2007 in the Cochrane Central Register of Controlled Trials; third, we screened 48 pediatric meta-analyses published in the Cochrane Database of Systematic Reviews in 2011. We extracted information on age ranges and age-subgroups considered and age-subgroup differences reported.

RESULTS

The age range of children in RCTs published in Pediatrics varied from 0.1 to 17.5 years (median age: 5; interquartile range: 1.8-10.2) and only 25% of those presented age-subgroup analyses. Large variability was also detected for age ranges in 188 RCTs from the Cochrane Central Register of Controlled Trials, and only 28 of those analyzed age-subgroups. Moreover, only 11 of 48 meta-analyses had age-subgroup analyses, and in 6 of those, only different studies were included. Furthermore, most of these observed differences were not beyond chance.

CONCLUSIONS

We observed large variability in the age ranges and age-subgroups of children included in recent pediatric trials and meta-analyses. Despite the limited available data, some age-subgroup differences were noted. The rationale for the selection of particular age-subgroups deserves further study.

Quality of registration for clinical trials published in emergency medicine journals

STUDY OBJECTIVE

In 2005, the International Committee of Medical Journal Editors established clinical trial registration as a requirement for articles submitted to member journals, with the goal of improving the transparency of clinical research. The objective of this study is to characterize the registration of clinical trials published in emergency medicine journals.

METHODS

Randomized trials involving human subjects and published between June 1, 2008, and May 31, 2011 in the 5 emergency medicine journals with the highest impact factors were included. We assessed the clarity of registered primary outcomes, timing of registration relative to patient enrollment, and consistency between registered and published outcomes.

RESULTS

Of the 123 trials included, registry entries were identified for 57 (46%). Of the 57 registered studies, 45 (79%) were registered after the initiation of subject enrollment, 9 (16%) had registered outcomes that were unclear, and 26 (46%) had discrepancies between registered and published outcomes. Only 5 studies were registered before patient enrollment with a clear primary outcome that was consistent with the published primary outcome. Annals of Emergency Medicine was the only journal in which the majority of trials were registered.

CONCLUSION

Current compliance with clinical trial registration guidelines is poor among trials published in emergency medicine journals.

StaR Child Health: developing evidence-based guidance for the design, conduct, and reporting of pediatric trials

Standards for Research in (StaR) Child Health was founded in 2009 to address the paucity and shortcomings of pediatric clinical trials. This initiative involves international experts who are dedicated to developing practical, evidence-based standards to enhance the reliability and relevance of pediatric clinical research. Through a systematic "knowledge to action" plan, StaR Child Health will make efforts to improve and expand the evidence base for child health across the world.

Quality of reporting of neonatal and infant trials in high-impact journals

OBJECTIVES

To perform a systematic review of the quality of reporting for randomized controlled trials (RCTs) with infants and neonates that were published in high-impact journals and to identify RCT characteristics associated with quality of reporting.

METHODS

RCTs that enrolled infants younger than 12 months and were published in 2005-2009 in 6 pediatric or general medical journals were reviewed. Eligible RCTs were evaluated for the presence of 11 quality criteria selected from the Consolidated Standards of Reporting Trials guidelines. The relationships between quality of reporting and key study characteristics were tested with nonparametric statistics.

RESULTS

Two reviewers had very good agreement regarding the eligibility of studies (κ = 0.85) and the presence of quality criteria (κ = 0.82). Among 179 eligible RCTs, reporting of the individual quality criteria varied widely. Only 50% included a flow diagram, but 99% reported the number of study participants. Higher quality of reporting was associated with greater numbers of study participants, publication in a general medical journal, and greater numbers of centers (P < .0001 for each comparison). Geographic region and positive study outcomes were not associated with reporting quality.

CONCLUSIONS

The quality of reporting of infant and neonatal RCTs is inconsistent, particularly in pediatric journals. Therefore, readers cannot assess accurately the validity of many RCT results. Strict adherence to the Consolidated Standards of Reporting Trials guidelines should lead to improved reporting.