Developing a scoring guide for the Appraisal of Guidelines for Research and Evaluation II instrument in Korea: a modified Delphi consensus process

Characterization and critical appraisal of physiotherapy intervention research in Nigeria: a systematic review

OBJECTIVES

Clinical research is the bedrock of clinical innovation, education and practice. We characterized and critically appraised physiotherapy clinical research to avoid implementing misleading research findings into practice and to task the Nigerian physiotherapy societies on responsible conduct of clinical research.

METHODS

This is a systematic review of articles published in English between 2009 and 2023. We started with 2009 because at least few Nigerian Physiotherapy school had commenced postgraduate (research) training by then. We searched Pubmed, Medline, Cumulative Index to Nursing and Allied Health Literature, Academic Search Complete, PsycINFO and African Journal Online, and reference lists of relevant articles. We Data were selected and extracted according to predesigned eligibility criteria and using a standardized data extraction table. Where appropriate, the Pedro and Cochrane ROBINS1 were used to examine the risk of bias.

RESULTS

A total of 76 Nigerian studies were included in this study. The mean age of the study participants was 46.7 ± 8.6 years. Approximately, 45% of the participants were males. Of the clinical experiments, the randomized controlled trial (RCT) was the most common design (87.5%). Musculoskeletal conditions (39.3%) were the most studied disorder. Approximately 86% of the RCT had studies possessed fair to good quality. Interventions constituted exercise therapy (76.3%), manual therapy (8.5%) and electrotherapy (8.5%). More than half (67.8%) of the studies recorded medium to large effect sizes. A fair proportion (48.2%) of the studies had a confounding-by-indication bias. Approximately 43% of the clinical experiments were underpowered, and a few studies conducted normality tests (10.9%) and intention-to-treat analysis (37.5%).

CONCLUSIONS

RCT is the most frequent clinical experiment, with majority of them possessing fair to good quality. The most important flaws include improper computation of sample size, statistical analysis, absent intention-to-treat approach, among others. The magnitude of effects of Physiotherapy interventions varies from nil effect to large effect. Musculoskeletal condition is the most prevalent disorder and exercise is the most important intervention in Nigerian physiotherapy practice.

TRIAL REGISTRATION

We registered the protocol with PROSPERO. The registration number: CRD42021228514.

History of the Task Force for the Korean Clinical Guidelines of the Developmental Disorders

Under the Ministry of Health and Welfare of the Republic of Korea, the National Autism and Developmental Disorder Centers for people with developmental disabilities are gradually expanding. The headquarters of the National Autism and Developmental Disorder Center provides support for education, training, and research, and several centers have been effectively operating since 2020. This study aimed to provide practical recommendations and guidelines for specialists such as clinical psychologists, child psychiatrists, allied professionals, community workers, and related administrators. It was developed as a guideline to promote early diagnosis, provide important information on integrated treatment, and assist people with developmental disabilities in Korea to make the best decisions for their quality of life.

Korean Clinical Practice Guidelines: Current Status of Adherence to the RIGHT Checklist

BACKGROUND

The Korean Academy of Medical Sciences (KAMS) has been utilizing AGREE II to audit the quality of clinical practice guidelines (CPGs) developed in Korea. Monitoring the RIGHT Checklist adherence could help monitor the quality status and discover areas for improvement of CPG development.

METHODS

We included 129 CPGs from the past 5 years and assessed each item of the RIGHT Checklist. STATA version 15.0 was used for statistical analysis.

RESULTS

Among the seven sections of the RIGHT checklist, sections with a full compliance rate over 60% were 'basic information' (65%) and 'background' (66%). The other sections' mean full compliance rates were 'Evidence' 52%, 'Recommendation' 35%, 'Review and quality assurance' 25% and 'Funding, declaration and management of interest' 17%. Sections with a partial compliance rate over 60% were 'Recommendation' (60%) and 'Funding, declaration and management of interest' (70%). Non-compliance was highest in the 'Review and quality assurance' (17%) domain. In comparison between groups 1 (under median group) and 2 (over median group), group 2 showed a tendency to have multi-stakeholder involvement and present sufficient information on financial resources and conflict of interest declarations. For the CPGs developmental methodology aspect, group 2 provided more pertinent information than group 1 about supporting evidence-making and the process from evidence to recommendation.

CONCLUSION

This study evaluated adherence to the RIGHT Checklist of CPGs developed in Korea. It can provide helpful information to develop strategic plans for enhancing the capabilities of developing CPGs in Korea.

We Agree, Don't We? The Delphi Method for Health Environments Research

OBJECTIVE

This overview is intended to provide the process framework for built environment researchers to use the Delphi method. The article outlines the methodological criteria originally established for the Delphi method, as well as commonly accepted modifications, to advance guidance for evidence-based built environment considerations.

BACKGROUND

Increasingly used in healthcare research, the Delphi method is a process for gaining consensus through controlled feedback from a panel-a group made up of experts or individuals knowledgeable on the subject. The method is often used where there is limited or conflicting evidence, where participants may be geographically dispersed, and where anonymity is desired to control for dominant individuals. The Delphi method consists of panel selection, development of content surveys, and iterative stages of anonymous responses to gain consensus. Panelists receive feedback after each round in the form of a statistical representation of the overall group's response. The goal of multiple iterations in the Delphi method is to reduce the range of responses and gain expert consensus, which is often seen as more credible than conjecture or individual opinion.

CONCLUSION

With a geographic diversity of healthcare design expertise, and with so many aspects of healthcare design lacking a robust body of supporting empirical research, the Delphi method is well-suited to developing evidence-based design recommendations and considerations for healthcare built environments.

Revised Korean Society of Infectious Diseases/National Evidence-based Healthcarea Collaborating Agency Guidelines on the Treatment of Patients with COVID-19

Despite the global effort to mitigate the spread, coronavirus disease 2019 (COVID-19) has become a pandemic that took more than 2 million lives. There are numerous ongoing clinical studies aiming to find treatment options and many are being published daily. Some effective treatment options, albeit of variable efficacy, have been discovered. Therefore, it is necessary to develop an evidence-based methodology, to continuously check for new evidence, and to update recommendations accordingly. Here we provide guidelines on pharmaceutical treatment for COVID-19 based on the latest evidence.

Development of ICNP-based inpatient falls prevention catalogue

OBJECTIVES

The International Council of Nurses supports the development of International Classification for Nursing Practice® (ICNP®) catalogues to support the construction of electronic health records and evidence-based practice. Such a catalogue is needed for inpatient falls.

METHODS

The ICNP®-Based Inpatient Fall-Prevention Catalogue ('The Catalogue') was developed following the six steps recommended by the International Council of Nurses: (1) identifying inpatient falls as a priority, (2) gathering relevant concepts from 10 international guidelines and comparing locally defined sets of fall-prevention terms, (3) mapping the concepts to the ICNP® terminology, (4) identifying new concepts, (5) conducting a clinical face validation with a 12-member panel and finalizing The Catalogue and (6) setting a strategy for dissemination. The high-level structure of the International Classification for Patient Safety was used as a theoretical framework.

RESULTS

Eighteen nursing care elements and 141 terms were identified. A local vocabulary set had 89 terms (63.1%) that all corresponded to the identified terms. The exact and post-coordination mapping rates to the ICNP® were 75% and 40.6% for assessment/diagnosis/outcomes and interventions, respectively. The 54 new terms corresponded to 52 primitive concepts. An external review of The Catalogue showed that it had adequate understandability and validity. However, one-third of assessments/diagnoses/outcomes and one-fourth of interventions were not found in a tertiary hospital practice.

CONCLUSION

A fall-prevention catalogue has been developed based on evidence and a theoretical framework and also clinically validated.

IMPLICATION FOR NURSING AND HEALTH POLICY

The Catalogue is a standardized interface terminology and content subset in any electronic health records system that can directly deliver evidence on fall prevention. It can also be used as an informatics tool to aggregate, analyse, interpret and compare nursing data worldwide.

Delphi Survey for Designing a Intervention Research Study on Childhood Obesity Prevention

Background

The prevalence of childhood obesity in South Korea has increased owing to economic improvement and the prevailing Westernized dietary pattern. As the incidence of chronic diseases caused by obesity is also expected to increase, effective interventions to prevent childhood obesity are needed. Therefore, we conducted a Delphi study to determine the priorities of a potential intervention research on childhood obesity prevention and its adequacy and feasibility.

Methods

The two-round Delphi technique was used with a panel of 10 childhood obesity experts. The panelists were asked to rate “priority populations,” “methods of intervention,” “measurement of outcomes,” “future intervention settings,” and “duration of intervention” by using a structured questionnaire. Finally, a portfolio analysis was performed with the adequacy and feasibility indexes as the two axes.

Results

For priority populations, the panel favored “elementary,” “preschool,” and “middle and high school” students in this order. Regarding intervention settings, the panelists assigned high adequacy and feasibility to “childcare centers” and “home” for preschool children, “school” and “home” for elementary school children, and “school” for adolescents in middle and high school. As the age of the target population increased, the panelists scored increasing numbers of anthropometric, clinical, and intermediate outcomes as highly adequate and feasible for assessing the effectiveness of the intervention.

Conclusion

According to the results of the Delphi survey, the highest-priority population for the research on childhood obesity prevention was that of elementary school students. Various settings, methods, outcome measures, and durations for the different age groups were also suggested.

Methodological Quality Appraisal of 27 Korean Guidelines Using a Scoring Guide Based on the AGREE II Instrument and a Web-based Evaluation

This study evaluated the methodological quality of CPGs using the Korean AGREE II scoring guide and a web-based appraisal system and was conducted by qualified appraisers. A total of 27 Korean CPGs were assessed under 6 domains and 23 items on the AGREE II instrument using the Korean scoring guide. The domain scores of the 27 guidelines were as following: the mean domain score was 82.7% (median 84.7%, ranging from 55.6% to 97.2%) for domain 1 (scope and purpose); 53.4% (median 56.9%, ranging from 11.1% to 95.8%) for domain 2 (stakeholder involvement); 63.0% (median 71.4%, ranging from 13.5% to 90.6%) for domain 3 (rigor of development); 88.9% (median 91.7%, ranging from 58.3% to 100.0%) for domain 4 (clarity of presentation); 30.1% (median 27.1%, ranging from 3.1% to 67.7%) for domain 5 (applicability); and 50.2% (median 58.3%, ranging from 0.0% to 93.8%) for domain 6 (editorial independence). Three domains including scope and purpose, rigor of development, and clarity of presentation were rated at more than 60% of the scaled domain score. Three domains including stakeholder involvement, applicability, and editorial independence were rated at less than 60% of the scaled domain score. Finally, of the 27 guidelines, 18 (66.7%) were rated at more than 60% of the scaled domain score for rigor of development and were categorized as high-quality guidelines.

Korean clinical practice guideline for benign prostatic hyperplasia

In 2014, the Korean Urological Association organized the Benign Prostatic Hyperplasia Guideline Developing Committee composed of experts in the field of benign prostatic hyperplasia (BPH) with the participation of the Korean Academy of Family Medicine and the Korean Continence Society to develop a Korean clinical practice guideline for BPH. The purpose of this clinical practice guideline is to provide current and comprehensive recommendations for the evaluation and treatment of BPH. The committee developed the guideline mainly by adapting existing guidelines and partially by using the de novo method. A comprehensive literature review was carried out primarily from 2009 to 2013 by using medical search engines including data from Korea. Based on the published evidence, recommendations were synthesized, and the level of evidence of the recommendations was determined by using methods adapted from the 2011 Oxford Centre for Evidence-Based Medicine. Meta-analysis was done for one key question and four recommendations. A draft guideline was reviewed by expert peer reviewers and discussed at an expert consensus meeting until final agreement was achieved. This evidence-based guideline for BPH provides recommendations to primary practitioners and urologists for the diagnosis and treatment of BPH in men older than 40 years.

National Priority Setting of Clinical Practice Guidelines Development for Chronic Disease Management

By November 2013, a total of 125 clinical practice guidelines (CPGs) have been developed in Korea. However, despite the high burden of diseases and the clinical importance of CPGs, most chronic diseases do not have available CPGs. Merely 83 CPGs are related to chronic diseases, and only 40 guidelines had been developed in the last 5 yr. Considering the rate of the production of new evidence in medicine and the worsening burden from chronic diseases, the need for developing CPGs for more chronic diseases is becoming increasingly pressing. Since 2011, the Korean Academy of Medical Sciences and the Korea Centers for Disease Control and Prevention have been jointly developing CPGs for chronic diseases. However, priorities have to be set and resources need to be allocated within the constraint of a limited funding. This study identifies the chronic diseases that should be prioritized for the development of CPGs in Korea. Through an objective assessment by using the analytic hierarchy process and a subjective assessment with a survey of expert opinion, high priorities were placed on ischemic heart disease, cerebrovascular diseases, Alzheimer's disease and other dementias, osteoarthritis, neck pain, chronic kidney disease, and cirrhosis of the liver.

Development of Quality Management Systems for Clinical Practice Guidelines in Korea

This study introduces the Clinical practice guidelines (CPGs) appraisal system by the Korean Academy of Medical Sciences (KAMS). Quality management policies for CPGs vary among different countries, which have their own cultures and health care systems. However, supporting developers in guideline development and appraisals using standardized tools are common practices. KAMS, an organization representing the various medical societies of Korea, has been striving to establish a quality management system for CPGs, and has established a CPGs quality management system that reflects the characteristics of the Korean healthcare environment and the needs of its users. KAMS created a foundation for the development of CPGs, set up an independent appraisal organization, enacted regulations related to the appraisals, and trained appraisers. These efforts could enhance the ability of each individual medical society to develop CPGs, to increase the quality of the CPGs, and to ultimately improve the quality of the information available to decision-makers.

Firefighting to Innovation: Using Human Factors and Ergonomics to Tackle Slip, Trip, and Fall Risks in Hospitals

OBJECTIVE

The aim of this study was to use a theoretical model (bench) for human factors and ergonomics (HFE) and a comparison with occupational slips, trips, and falls (STFs) risk management to discuss patient STF interventions (bedside).

BACKGROUND

Risk factors for patient STFs have been identified and reported since the 1950s and are mostly unchanged in the 2010s. The prevailing clinical view has been that STF events indicate underlying frailty or illness, and so many of the interventions over the past 60 years have focused on assessing and treating physiological factors (dizziness, illness, vision/hearing, medicines) rather than designing interventions to reduce risk factors at the time of the STF.

METHOD

Three case studies are used to discuss how HFE has been, or could be, applied to STF risk management as (a) a design-based (building) approach to embed safety into the built environment, (b) a staff- (and organization-) based approach, and (c) a patient behavior-based approach to explore and understand patient perspectives of STF events.

RESULTS AND CONCLUSION

The results from the case studies suggest taking a similar HFE integration approach to other industries, that is, a sustainable design intervention for the person who experiences the STF event-the patient.

APPLICATION

This paper offers a proactive problem-solving approach to reduce STFs by patients in acute hospitals. Authors of the three case studies use HFE principles (bench/book) to understand the complex systems for facility and equipment design and include the perspective of all stakeholders (bedside).

Applying the RE-AIM Framework to Evaluate the Dissemination and Implementation of Clinical Practice Guidelines for Sexually Transmitted Infections

Clinical practice guidelines (CPG) are one of the most effective ways to translate evidence of medical improvement into everyday practice. This study evaluated the dissemination and implementation of the Sexually Transmitted Infections-Korean Guidelines (STIKG) by applying the reach, effectiveness, adoption, implementation and maintenance (RE-AIM) framework. A survey questionnaire was administered to clinicians via the internet. Among the 332 respondents, 190 (57.2%) stated that they were aware of STIKG and 107 (33.2%) implemented STIKG in their practice. The odds that a physician was exposed to STIKG (dissemination) were 2.61 times greater among physicians with previous training or education for any CPG than those who did not. Clinicians who indicated that STIKG were easy to understand were 4.88 times more likely to implement STIKG in their practice than those who found them not so easy. When a clinician's workplace had a supporting system for CPG use, the odds of implementation was 3.76 times higher. Perceived level of effectiveness of STIKG did not significantly influence their implementation. The findings of this study suggest that, ultimately, knowing how to engage clinicians in CPG implementation is as important as how to disseminate such guidelines; moreover, easy-to-use guidelines and institutional support are key factors.

Development of a guideline for the management of the unanticipated difficult airway in pediatric practice

BACKGROUND

Most airway problems in children are identified in advance; however, unanticipated difficulties can arise and may result in serious complications. Training for these sporadic events can be difficult. We identified the need for a structured guideline to improve clinical decision making in the acute situation and also to provide a guide for teaching.

OBJECTIVE

Guidelines for airway management in adults are widely used; however, none have been previously devised for national use in children. We aimed to develop guidelines for the management of the unanticipated difficult pediatric airway for use by anesthetists working in the nonspecialist pediatric setting.

METHOD

We reviewed available guidelines used in individual hospitals. We also reviewed research into airway management in children and graded papers for the level of evidence according to agreed criteria. A Delphi panel comprising 27 independent consultant anesthetists considered the steps of the acute airway management guidelines to reach consensus on the best interventions to use and the order in which to use them. If following the literature review and Delphi feedback, there was insufficient evidence or lack of consensus, regarding inclusion of a particular point; this was reviewed by a Second Specialist Group comprising 10 pediatric anesthetists.

RESULTS

Using the Delphi group's deliberations and feedback from the Second Specialist Group, we developed three guidelines for the acute airway management of children aged 1-8 years.

CONCLUSIONS

This paper provides the background, available evidence base, and justification for each step in the resultant guidelines and gives a rationale for their use.

Improving the reliability of clinical practice guideline appraisals: effects of the Korean AGREE II scoring guide

The Korean translated Appraisal of Guidelines for Research and Evaluation II (Korean AGREE II) instrument was distributed into Korean medical societies in 2011. However, inter-rater disagreement issues still exist. The Korean AGREE II scoring guide was therefore developed to reduce inter-rater differences. This study examines the effects of the Korean AGREE II scoring guide to reduce inter-rater differences. Appraisers were randomly assigned to two groups (Scoring Guide group and Non-Scoring Guide group). The Korean AGREE II instrument was provided to both groups. However, the scoring guide was offered to Scoring Guide group only. Total 14 appraisers were participated and each guideline was assessed by 8 appraisers. To evaluate the reliability of the Korean AGREE II scoring guide, correlation of scores among appraisers and domain-specific intra-class correlation (ICC) were compared. Most scores of two groups were comparable. Scoring Guide group showed higher reliability at all guidelines. They showed higher correlation among appraisers and higher ICC values at almost all domains. The scoring guide reduces the inter-rater disagreement and improves the overall reliability of the Korean-AGREE II instrument.