Online Modified-Delphi: a Potential Method for Continuous Patient Engagement Across Stages of Clinical Practice Guideline Development

Development of a Maternal Health Care Exercise and Pelvic Abdominal Rehabilitation Curriculum for Obstetric Medical Professionals Using a Modified Delphi Process

INTRODUCTION AND HYPOTHESIS

The purpose of this study was to develop a maternal health care exercise and pelvic abdominal rehabilitation curriculum for obstetric medical professionals. Maternal health care exercise and pelvic abdominal rehabilitation can effectively prevent and alleviate pelvic-floor-rehabilitation problems in pregnant women and improve their quality of life. However, there is a lack of standardized courses for training obstetric medical professionals.

METHODS

This study used the Delphi method to design two rounds of expert consultation. A group of 17 specialists from the fields of gynecology, obstetrics, and rehabilitation, who are involved in clinical teaching and management, participated in the study. Data were collected between October and December 2023.

RESULTS

The effective recovery rates of the two rounds of questionnaires were 85.0% and 94.1%. The expert-authority coefficients were 0.90 and 0.88. The Kendall coordination coefficient was 0.35-0.41. The average importance score for each index ranged from 3.25 to 4.94. The coefficient of variation for each index ranged from 0.07 0.20. Based on the curricular topics, we constructed four first-level indicators, 16 second-level indicators, and 58 third-level indicators of exercise and pelvic and abdominal rehabilitation for obstetric medical professionals.

CONCLUSION

The training curriculum provides obstetric medical professionals with the basis for obstetric rehabilitation. The training system is clear and coherent, and the content is clear and comprehensive, which enriches global obstetrical rehabilitation education resources. It is helpful to guide and educate on maternal obstetric rehabilitation to improve the quality of life of the women.

Seven principles for integrating health equity considerations in the practice guideline enterprise

BACKGROUND AND OBJECTIVES

Health equity aims to provide all individuals with equal and fair opportunities to achieve optimal health. Practice guidelines can play a pivotal role in advancing health equity; yet, few organizations use tools to systematically integrate health equity considerations. Thus, it is important to establish a foundation for practical tools to support the systematic integration of health equity considerations. This manuscript proposes principles for the integration of health equity considerations in the practice guideline enterprise.

METHODS

In the process of developing an equity extension for the GIN-McMaster guideline development checklist, we established a diverse advisory group of guideline developers, patients, members of the public experiencing inequities, health equity researchers, and guideline developers. We formulated the principles informed by a methodological review of guideline handbooks and iterative discussions between working group members.

RESULTS

We identified the following seven principles for integrating health equity considerations into the practice guideline enterprise: (1) articulating health equity, (2) a priori planning for considering health equity, (3) selection and engagement of individuals with lived experiences of inequities, (4) equity in evidence synthesis, (5) developing equity-informed recommendations, (6) inclusive knowledge mobilization, and (7) evaluating the impact of health equity considerations. We elaborated on the importance of the principles using published examples and mapped them to the different phases of the guideline development process.

CONCLUSION

Guideline developers should adhere to these principles in the development of guidelines and health equity guideline development and appraisal tools. These principles are the foundational concepts for developing health equity extension items for the GIN-McMaster guideline development checklist.

The technical expert/clinical user/patient panel (TECUPP): centering patient and family perspectives in patient-reported measure development

Patient and caregiver perspectives are increasingly incorporated into health care research and policymaking, but their inclusion in the quality measure development process often is not robust. We describe a stakeholder panel model for incorporating patient/caregiver voices in the development of patient-reported measures, the Technical Expert/Clinical User/Patient Panel (TECUPP) model. This model is characterized by significant or equal representation of people with lived experience of the disease or condition (as patients or caregivers) to the clinicians and others with technical expertise who typically comprise technical expert panels. We report key design features of the TECUPP model and how we used this model to develop survey-based patient experience measures of timeliness of cancer diagnosis. We describe benefits and challenges of using the TECUPP model and considerations to guide others who might use it as part of developing patient-centered quality measures, based on our experience convening a TECUPP to inform development of a patient-reported measure on timeliness of cancer diagnosis. Benefits include creating space for significant contributions from patients/caregivers and development of a shared understanding of patient experiences and observability of measure domains between clinicians and patients/caregivers. Challenges include time management and managing conversations outside the project scope. Measure development efforts implementing this model should consider recruiting diverse individuals, scheduling short and frequent meetings, enabling participation from all TECUPP members, developing accessible pre-read materials, anchoring meetings with patient stories, and encouraging multiple communication modes. The TECUPP model promotes discussion and understanding by patients/caregivers and clinicians/measure experts helpful for development of survey-based patient-reported measures.

Guidance for engagement in health guideline development: A scoping review

Background

Health guideline developers engage with interested people and groups to ensure that guidelines and their recommendations are relevant and useful to those who will be affected by them. These 'interest-holders' include patients, payers/purchasers of health services, payers of health research, peer review editors, product makers, programme managers, policymakers, providers, principal investigators, and the public. The Guidelines International Network (GIN) and McMaster University Guideline Development Checklist describes 146 steps of the guideline process organized into 18 topics. While one topic focuses on engagement, it does not describe how to engage with interest-holders. In addition, interest-holder input could be sought throughout the guideline development process. This scoping review is part of a series of four related reviews. The three other reviews address barriers and facilitators to engagement in guideline development, managing conflicts of interest in guideline development, and assessing the impact of interest-holder engagement on guideline development. The four reviews will inform the development of guidance for multi-interest-holder engagement in guideline development; the GIN-McMaster Guideline Development Checklist Extension for Engagement.

Objectives

The objective of this scoping review is to identify, describe, and summarise existing guidance and methods for multi-interest-holder engagement throughout the health guideline development process.

Search Methods

We conducted one comprehensive search for studies of engagement in guidelines to meet the inclusion criteria of one or more of the four systematic reviews in this series. We searched MEDLINE (OVID), CINAHL (EBSCO), EMBASE (OVID), PsycInfo (OVID) and SCOPUS databases up to September 2022. We did not include limits for date, study design, or language. We searched websites of agencies and organizations that engage interest-holder groups, such as the Agency for Healthcare Research and Quality (AHRQ), CIHR Strategy for Patient-Oriented Research (SPOR), National Institute for Health and Care Research (NIHR) Be Part of Research, Guidelines International Network (G-I-N), the National Institute for Health and Care Excellence, and the PatientCentred Outcomes Research Institute (PCORI). We handsearched the websites of guideline producing agencies. We solicited additional grey literature from the members of the MuSE Consortium.

Selection Criteria

Studies were included in this review if they reported on engagement of any of our identified groups, patients, payers/funders of research, payers/purchasers of health services, policymakers, programme managers, providers, principal investigators/researchers, peer review editors, product makers in the development of a health guideline. Titles and abstracts of identified citations were screened independently, in duplicate. The full text of potentially relevant papers were screened for eligibility into one or more of the four reviews in the series. Screening was done independently, by two reviewers. The team held weekly meetings with all reviewers involved in screening to discuss and resolve conflicts.

Data Collection and Analysis

Two reviewers extracted relevant data into a pilot-tested data extraction form using Excel. We used the GIN-McMaster guideline development checklist as a framework for extracting the available guidance for each of our identified interest-holder groups throughout the development process. We presented descriptive statistics of the number of papers reporting guidance for each groups across the steps of the guideline process. We synthesized the relevant text using a qualitative meta-summary approach.

Main Results

We included 16 papers (from 17 reports). These papers were from Australia, Denmark, the Netherlands, the UK, and the USA, and eight papers were international (countries not specified). The papers provided guidance for at least one of our interest-holder groups for at least one stage of guideline development. We mapped this guidance to the GIN-McMaster Guideline Development Checklist to identify the available guidance for each of our interest-holder groups across all stages of the guideline development process. Guidance was available for patient engagement in 15 of the 16 papers. At least two papers provided guidance for each of the 18 topics of the GIN-McMaster Guideline Development Checklist. For healthcare providers, 9 papers provided guidance for their engagement across 10 of the 18 guideline development topics. Guidance for engaging with the public was provided for 14 of the 18 topics and reported in 4 of our included papers. For payers/purchasers of health services, policymakers, product makers, programme managers, and principal investigators, 2-3 papers provided guidance for these groups across 4-7 topics of the GIN-McMaster checklist. We did not identify any specific guidance for payers of health research or for editors of peer-reviewed journals.

Authors' Conclusions

Guidance for interst-holder engagement in guidelines is available but has focused primarily on patients. We will utilize the guidance identified in this scoping review to inform the GIN-McMaster Guideline Development Checklist Extension for engagement. Combined with the information obtained from the other systematic reviews in this series, we will address the gaps in guidance for the other identified interest-holder groups.

Assessing parent-child interaction with deaf and hard of hearing infants aged 0-3 years: An international multi-professional e-Delphi

INTRODUCTION

Most deaf babies are born to hearing families who do not yet have the skills to communicate effectively with their child. Adaptations to communication are important because the quality of parent-child interaction (PCI) predicts how a deaf child develops language. Teachers of Deaf children and Speech and Language Therapists support families with communication in the home. Currently, there are no assessments that appraise how a parent interacts with their deaf baby. Previous research has identified which parent behaviours and approaches are used in PCI assessments in research and practice. The current paper forms consensus on the core content and best practices of a new PCI tool for deaf children aged 0-3 years.

METHODS

An international sample of expert academics and practitioners (n = 83) were recruited to take part in a two-round modified electronic Delphi study. Participants were presented with 69 statements focusing on (i) which parent behaviours were important in assessment (ii) the methods to be used in PCI assessment. Participants rated the extent to which they agreed or disagreed with each statement on a five-point Likert scale and gave comments to support their response. Consensus was defined as >80% of participants rating the statement as a (4) 'highly important' or a (5) 'essential'. If consensus was not reached, participant comments were used to generate new statements which were rated in the second round. This project involved a patient and public involvement (PPI) group of hearing and deaf parents and professionals to design and guide the study.

RESULTS

Consensus was achieved on 52 statements and ranged from 80-99%. A further six statements were additionally included. Within the 58 statements included, 36 were parent behaviours which centred on the parent's observation of, and response to, their child's behaviour and/or language. The remaining 22 statements focused on methods used in the assessment such as parents having their PCI filmed, parents having the opportunity to review the video and assess themselves alongside a professional, and parents being involved in subsequent goal setting.

CONCLUSIONS

This e-Delphi presented the parent behaviours and methods of assessment to be included in a new PCI tool for deaf children. Future co-production work and acceptability and feasibility testing are discussed.

Disciplinary trends in the use of the Delphi method: A bibliometric analysis

The Delphi method is an iterative, anonymous, group-based process for eliciting and aggregating opinion on a topic to explore the existence of consensus among experts. The year 2023 marks the 60th anniversary of the first peer-reviewed journal article on the Delphi method. Originally developed for operations research, this method is now applied extensively by researchers representing diverse scientific fields. We used a bibliometric analysis to describe general trends in the expansion of its use across disciplines over time. We conducted a systematic literature search for all English-language, peer-reviewed journal articles on the Delphi method through its first 60 years. We found 19,831 articles: 96.8% (n = 19,204) on the actual use of the Delphi method in an empirical study and 3.2% (n = 627) describing, examining, or providing some guidance on how to use the Delphi method. Almost half (49.9%) of all articles were published in the 2010s and an additional third (32.5%) in the first few years of the 2020s. Nearly two-thirds (65%, n = 12,883) of all published articles have appeared in medical journals, compared to 15% in science and technology (n = 3,053) or social science (n = 3,016) journals. We conclude that the expanded use of the Delphi method has been driven largely by the medical field, though social scientists and technologists continue to be at the forefront of methodological work on the Delphi method. Therefore, we call for greater transdisciplinary collaboration on methodological guidance and standards for the Delphi method.

European e-Delphi process to define expert consensus on electrochemotherapy treatment indications, procedural aspects, and quality indicators in melanoma

BACKGROUND

Skin metastases are an important co-morbidity in melanoma. Despite broad adoption, electrochemotherapy implementation is hindered by a lack of treatment indications, uncertainty regarding procedural aspects, and the absence of quality indicators. An expert consensus may harmonize the approach among centres and facilitate comparison with other therapies.

METHODS

An interdisciplinary panel was recruited for a three-round e-Delphi survey. A literature-based 113-item questionnaire was proposed to 160 professionals from 53 European centres. Participants rated each item for relevance and degree of agreement on a five-point Likert scale, and received anonymous controlled feedback to allow revision. The items that reached concordant agreement in two successive iterations were included in the final consensus list. In the third round, quality indicator benchmarks were defined using a real-time Delphi method.

RESULTS

The initial working group included 122 respondents, of whom 100 (82 per cent) completed the first round, thus qualifying for inclusion in the expert panel (49 surgeons, 29 dermatologists, 15 medical oncologists, three radiotherapists, two nurse specialists, two clinician scientists). The completion rate was 97 per cent (97 of 100) and 93 per cent (90 of 97) in the second and third rounds respectively. The final consensus list included 54 statements with benchmarks (treatment indications, (37); procedural aspects, (1); quality indicators, (16)).

CONCLUSION

An expert panel achieved consensus on the use of electrochemotherapy in melanoma, with a core set of statements providing general direction to electrochemotherapy users to refine indications, align clinical practices, and promote quality assurance programmes and local audits. The residual controversial topics set future research priorities to improve patient care.

Moving toward standardization: physician reporting of sleep studies

Detailed primary data collected from sleep studies should lead to specific and clear reports with evidence-based clinical recommendations that, when introduced by sleep medicine specialists, create a window of opportunity to support our non-sleep medicine referring teams and to engage patients and caregivers in their care as recipients of the reports. This is how sleep study reporting differs from other test reports; currently, there is wide variation in how the data collected are presented and summarized. The goal of this document is to offer recommendations for structured reporting of sleep studies. We offer a practical, complete, and relevant document and a structure that can be implemented across sleep centers nationwide and does not burden the interpreter. We anticipate some readers will opine that some of the content is beyond the scope of what the interpreter physician needs to include, while others will propose missing data that they feel should have been included. We feel that the flexibility of the proposal accommodates for this and allows for a "first step" toward standardization of physician reporting of sleep studies. High-quality structured reporting of sleep studies is becoming ever more important for patient care, benefiting patients, caregivers, clinicians, durable medical equipment companies, and payers.

CITATION

Lastra AC, Ingram D, Park J, et al. Moving toward standardization: physician reporting of sleep studies. J Clin Sleep Med. 2023;19(3):595-603.

Existing guidance on reporting of consensus methodology: a systematic review to inform ACCORD guideline development

OBJECTIVE

To identify evidence on the reporting quality of consensus methodology and to select potential checklist items for the ACcurate COnsensus Reporting Document (ACCORD) project to develop a consensus reporting guideline.

DESIGN

Systematic review.

DATA SOURCES

Embase, MEDLINE, Web of Science, PubMed, Cochrane Library, Emcare, Academic Search Premier and PsycINFO from inception until 7 January 2022.

ELIGIBILITY CRITERIA

Studies, reviews and published guidance addressing the reporting quality of consensus methodology for improvement of health outcomes in biomedicine or clinical practice. Reports of studies using or describing consensus methods but not commenting on their reporting quality were excluded. No language restrictions were applied.

DATA EXTRACTION AND SYNTHESIS

Screening and data extraction of eligible studies were carried out independently by two authors. Reporting quality items addressed by the studies were synthesised narratively.

RESULTS

Eighteen studies were included: five systematic reviews, four narrative reviews, three research papers, three conference abstracts, two research guidance papers and one protocol. The majority of studies indicated that the quality of reporting of consensus methodology could be improved. Commonly addressed items were: consensus panel composition; definition of consensus and the threshold for achieving consensus. Items least addressed were: public patient involvement (PPI); the role of the steering committee, chair, cochair; conflict of interest of panellists and funding. Data extracted from included studies revealed additional items that were not captured in the data extraction form such as justification of deviation from the protocol or incentives to encourage panellist response.

CONCLUSION

The results of this systematic review confirmed the need for a reporting checklist for consensus methodology and provided a range of potential checklist items to report. The next step in the ACCORD project builds on this systematic review and focuses on reaching consensus on these items to develop the reporting guideline.

PROTOCOL REGISTRATION

https://osf.io/2rzm9.

ACCORD guideline for reporting consensus-based methods in biomedical research and clinical practice: a study protocol

BACKGROUND

Structured, systematic methods to formulate consensus recommendations, such as the Delphi process or nominal group technique, among others, provide the opportunity to harness the knowledge of experts to support clinical decision making in areas of uncertainty. They are widely used in biomedical research, in particular where disease characteristics or resource limitations mean that high-quality evidence generation is difficult. However, poor reporting of methods used to reach a consensus - for example, not clearly explaining the definition of consensus, or not stating how consensus group panellists were selected - can potentially undermine confidence in this type of research and hinder reproducibility. Our objective is therefore to systematically develop a reporting guideline to help the biomedical research and clinical practice community describe the methods or techniques used to reach consensus in a complete, transparent, and consistent manner.

METHODS

The ACCORD (ACcurate COnsensus Reporting Document) project will take place in five stages and follow the EQUATOR Network guidance for the development of reporting guidelines. In Stage 1, a multidisciplinary Steering Committee has been established to lead and coordinate the guideline development process. In Stage 2, a systematic literature review will identify evidence on the quality of the reporting of consensus methodology, to obtain potential items for a reporting checklist. In Stage 3, Delphi methodology will be used to reach consensus regarding the checklist items, first among the Steering Committee, and then among a broader Delphi panel comprising participants with a range of expertise, including patient representatives. In Stage 4, the reporting guideline will be finalised in a consensus meeting, along with the production of an Explanation and Elaboration (E&E) document. In Stage 5, we plan to publish the reporting guideline and E&E document in open-access journals, supported by presentations at appropriate events. Dissemination of the reporting guideline, including a website linked to social media channels, is crucial for the document to be implemented in practice.

DISCUSSION

The ACCORD reporting guideline will provide a set of minimum items that should be reported about methods used to achieve consensus, including approaches ranging from simple unstructured opinion gatherings to highly structured processes.