Improving the question formulation in Delphi‐like surveys: Analysis of the effects of abstract language and amount of information on response behavior

How Delphi studies in the health sciences find consensus: a scoping review

BACKGROUND

Delphi studies are primarily used in the health sciences to find consensus. They inform clinical practice and influence structures, processes, and framework conditions of healthcare. The practical research-how Delphi studies are conducted-has seldom been discussed methodologically or documented systematically. The aim of this scoping review is to fill this research gap and to identify shortcomings in the methodological presentation in the literature. On the basis of the analysis, we derive recommendations for the quality-assured implementation of Delphi studies.

METHODS

Forming the basis of this scoping review are publications on consensus Delphi studies in the health sciences between January 1, 2018, and April 21, 2021, in the databases Scopus, MEDLINE via PubMed, CINAHL, and Epistemonikos. Included were publications in German and English containing the words "Delphi" in the title and "health" and "consensus" in the title or abstract. The practical research was analyzed for the qualitative content of the publications according to three deductive main categories, to which an influence on the result of Delphi studies can be imputed (expert panel, questionnaire design, process and feedback design).

RESULTS

A total of 287 consensus Delphi studies were included in the review, whereby 43% reported having carried out a modified Delphi. In most cases, heterogeneous expert groups from research, clinical practice, health economics, and health policy were surveyed. In about a quarter of the Delphi studies, affected parties, such as patients, were part of the expert panel. In the Delphi questionnaires it was most common for standardized Likert scales to be combined with open-ended questions. Which method was used to analyze the open-ended responses was not reported in 62% of the Delphi studies. Consensus is largely (81%) defined as percentage agreement.

CONCLUSIONS

The results show considerable differences in how Delphi studies are carried out, making assessments and comparisons between them difficult. Sometimes an approach points to unintended effects, or biases in the individual judgments of the respondents and, thus, in the overall results of Delphi studies. For this reason, we extrapolate suggestions for how certain comparability and quality assurance can be achieved for Delphi studies.

A consensus definition of creativity in surgery: A Delphi study protocol

INTRODUCTION

Clear definitions are essential in science, particularly in the study of abstract phenomena like creativity. Due to its inherent complexity and domain-specific nature, the study of creativity has been complicated, as evidenced by the various definitions used to describe it and the multitude of tools which claim to measure it. Surgery is a safety-critical profession where creativity could be useful in navigating unforeseen problems and circumstances, as well as developing new innovations to improve patient outcomes. To validly and reliably study creativity in surgery, a surgery-specific definition is required. We aim to develop a consensus definition of creativity in surgery, utilizing the existing creativity literature and surgeon input.

METHODS AND ANALYSIS

The objective of this study is to generate a consensus definition of creativity in surgery. We will first conduct a focus group comprised of 4-12 highly experienced surgeons to generate knowledge on surgeons' perceptions, attitudes and beliefs about creativity in surgery, collect real-world examples of creativity in surgery, and obtain opinions on the existing definitions of creativity in the literature. The selection of focus group participants will be performed using purposive sampling of the chairs and/or chiefs of each surgical sub-specialty at our home institution. Several questions relating to creativity in surgery will be posed to the focus group, to be rated using a 7-point Likert scale and used as prompts for group discussion. We will also search MEDLINE, PsycINFO and EMBASE to find definitions of creativity in the scientific literature. Six definitions, chosen based on citation frequency and relevancy to surgery, will be presented to the focus group for ranking and discussion. Lastly, in addition to novelty and effectiveness, which are widely accepted as necessary components of creativity, the focus group will be asked to consider the necessity of other components for creativity in surgery, sourced from the scientific literature. Descriptive and thematic analyses are planned for the quantitative and qualitative data, respectively. The results of the focus group will be incorporated in the drafting of five definitions of creativity in surgery, which will be presented as initial Delphi statements in the Delphi study. For the Delphi panel, we will perform non-probability purposive sampling of surgeons and surgeon trainees at our home institution, with a minimum panel size of 12. Panellists will be asked to select the definition of creativity most relevant to surgery, with each Delphi round electronically delivered. After each round, the steering committee will meet to review the results and adjust the statements for the next round based on the feedback. A maximum of 5 rounds will be performed, or until consensus is reached (≥75% agreement). Recruitment is scheduled to begin on 1 August 2024.

ETHICS AND DISSEMINATION

All focus group and panellists will be given written and verbal information on the study and provide signed, informed consent. We plan to publish the results of our study in a creativity science- or surgery-focused journal to disseminate the results of our study to relevant stakeholders. We also plan to present the results of our research at local, national, and international conferences.

Lost in translation: the lack of agreement between surgeons and scientists regarding biomaterials research and innovation for treating bone defects

BACKGROUND

With over 2 million grafts performed annually, bone ranks second only to blood in the frequency of transplants. This high demand is primarily driven by the persistent challenges posed by bone defects, particularly following trauma or surgical interventions such as tumour excision. The demand for effective and efficient treatments has increased exponentially in the twenty-first century. Limitations associated with autologous bone grafts drive exploration into replacements, including allografts, synthetic substitutes, and 3D-printed scaffolds. This research aimed to unravel disparities in the knowledge and evaluation of current and future bone defect treatments between surgeons and biomaterial scientists.

METHODS

A prospective cross-sectional survey, pre-registered with the OSF ( https://osf.io/y837m/?view_only=fab29e24df4f4adf897353ac70aa3361 ) and conducted online from October 2022 to March 2023, collected data on surgeons' views (n = 337) and scientists (n = 99) on bone defect treatments.

RESULTS

Scientists were significantly more optimistic than surgeons regarding the future replacement of autologous bone grafts with synthetic or tissue-engineered substitutes (p < 0.001). Accordingly, scientists foresee a paradigm shift from autologous bone grafts to biomaterial and tissue-engineered solutions, reflecting their confidence in the ongoing advancements within this field. Furthermore, regulatory trepidations for 3D-printed bone scaffolds were acknowledged, with scientists emphasizing the need for a more significant focus on clinical relevance in preclinical studies and regulatory clarity. In a ranked categorical assessment, witnessing the technology in action was deemed most influential in adopting new bone regeneration methods by both scientists and surgeons.

CONCLUSIONS

To conclude, this study was conducted through a web-based survey, highlighting a substantial translational gap. It underscores the immediate need ("call to action") for meaningful interdisciplinary collaboration between surgeons and scientists, often referred to as the need to "walk the talk". The findings underscore the critical importance of aligning clinical needs, research outcomes, and regulatory frameworks to improve the development and implementation of biomaterial-based bone graft substitutes that demonstrate efficacy and efficiency in bone defect treatment.

Determinants of First-Line Nurse Managers' Span of Control: A Delphi Study

Aims: The main goal of this research is to identify, through expert consensus, the key factors that determine the span of control (SOC) of first-line nurse managers (FLNMs) in the Spanish healthcare system. Background: The SOC is a management concept which has usually been defined as the number of subordinates reporting to a superior. In nursing, however, it is much more complex. This complexity is shaped by various factors related to patients, healthcare professionals and organisational structures. Nursing leaders must thoroughly consider these factors and their determinants, which necessitate a comprehensive assessment. Given the significant impact an inadequate SOC can have on patients, professionals and the organisation, it would be beneficial to address this issue. In nursing, studies on this subject are practically nonexistent and focus solely on the number of subordinates, highlighting the need for research in this area. Methods: Between September and December 2022, a Delphi study was conducted. Forty-five experts were invited to participate. The study involved nurse administrators, FLNMs, university professors, renowned researchers and other non-health professionals related to health management. The participants completed an online survey over three phases. Factor analysis was performed on the items for which consensus was reached. The jamovi software version 2.3.15 was used for data analysis. Results: A total of 35 experts participated in at least one of the three phases of the Delphi study. Following a comprehensive analysis of the identified factors, a consensus was reached on 31 of them. These were subsequently grouped into four categories: unit category (16 items, including complexity, resource management, conflicts and protocolisation and monitoring of activities), professional category (five items: number of staff, staffing stability and skill level and diversity of staff), FLNM category (four items: autonomy, experience and education and leadership style) and organisation category (six items: digitisation and information systems, education, research and implementation evidence-based practice and performing guards). Conclusions: Our research shows a high degree of consensus amongst participants in identifying the determinants and degree of relevance of SOC-related aspects. Although SOC is not currently assessed, all stakeholders agree that there are a large number of variables that should be considered when appointing a FLNM. Implications for Nurse Managers: Nursing managers can better assess the health of the organisation and improve performance by understanding the factors that influence the SOC of FLNMs. Due to the lack of previous studies, understanding these factors will allow the development of methods and tools tailored to the characteristics of different health systems.

Designing Entrustable Professional Activities for Treatment Planning of Oral Cancer by Maxillofacial Surgery Residents: A Modified Delphi Study

Purpose

The aim of this study was to develop a competency framework based on entrustable professional activities (EPAs) in oral cancer management by postgraduate trainees in oral and maxillofacial surgery through expert consensus.

Materials and Methods

The study design was based on a modified Delphi technique and involved iterative online surveys with two rounds of data collection and analysis. Initial development of the questionnaire identified five EPAs based on 42 competencies along with supervision level and assessment strategies. The first Delphi round involved administration of the survey questionnaire online to maxillofacial surgeons meeting the inclusion criteria for experts. Consensus was achieved on five EPAs and 36 competencies (≥80% response rate). Six competencies were rephrased and sent again in the Round 2 questionnaire to achieve a consensus.

Results

A total of 45 experts participated in Round 1 followed by input from 27 experts in Round 2 of the Delphi panel. Following two iterative rounds of online surveys and feedback, expert consensus was achieved to develop an EPA framework in five EPA domains focused on the management of oral cancer by postgraduate trainees in maxillofacial surgery including 38 specific competencies, supervision level, and assessment strategies. High content validity of the study was established through a comprehensive literature search, and expert feedback was evidenced by an excellent response rate (93.34%, and 64.28%) and a stringent criteria of response agreement amongst experts (≥80%).

Conclusion

In conclusion, this study employed expert consensus to identify five EPAs with 38 competencies along with the required supervision level of postgraduate maxillofacial trainees for the management of oral cancer. This EPA framework provides a roadmap for training supervisors to map the learning outcomes in oral oncology for postgraduate trainees in oral and maxillofacial surgery.

Macular hole Delphi consensus statement (MHOST)

PURPOSE

To derive a Delphi method-based consensus for the surgical management of Full Thickness Macular Hole (FTMH) and Lamellar Macular Hole (LMH).

METHODS

37 expert VR surgeons from 21 mainly European countries participated in Delphi method-based questionnaire for diagnosis and treatment of FTMHs and LMHs.

RESULTS

A total of 36 items were rated in round 1 by 37 participants, of which 10 items achieved consensus: intraoperative verification of PVD; clinical superiority of OCT-based FTMH classification; practical ineffectiveness of ocriplasmin; circular 360° ILM peeling for small macular holes; use of regular surgical technique for the size of the hole in concomitant retinal detachment; performing complete vitrectomy; SF6 gas as preferred tamponade; cataract surgery if crystalline lens is mildly/moderately opaque; removal of both ILM and LHEP in LMH surgery. In round 2, 18 items with moderate consensus (45-70% agreement) in round 1 were rated by 35 participants. Final consensus was reached in 35% of questions related to both diagnosis and surgical procedures.

CONCLUSIONS

This Delphi study provides valuable information about the consensus/disagreement on different scenarios encountered during FTMH and LMH management as a guide tosurgical decision-making. High rate of disagreement and/or variable approaches still exist for treating such relatively common conditions.

A Delphi Survey Study to Formulate Statements on the Treatability of Inherited Metabolic Disorders to Decide on Eligibility for Newborn Screening

The Wilson and Jungner (W&J) and Andermann criteria are meant to help select diseases eligible for population-based screening. With the introduction of next-generation sequencing (NGS) methods for newborn screening (NBS), more inherited metabolic diseases (IMDs) can technically be included, and a revision of the criteria was attempted. This study aimed to formulate statements and investigate whether those statements could elaborate on the criterion of treatability for IMDs to decide on eligibility for NBS. An online Delphi study was started among a panel of Dutch IMD experts (EPs). EPs evaluated, amended, and approved statements on treatability that were subsequently applied to 10 IMDs. After two rounds of Delphi, consensus was reached on 10 statements. Application of these statements selected 5 out of 10 IMDs proposed for this study as eligible for NBS, including 3 IMDs in the current Dutch NBS. The statement: 'The expected benefit/burden ratio of early treatment is positive and results in a significant health outcome' contributed most to decision-making. Our Delphi study resulted in 10 statements that can help to decide on eligibility for inclusion in NBS based on treatability, also showing that other criteria could be handled in a comparable way. Validation of the statements is required before these can be applied as guidance to authorities.

European consensus for the diagnosis of MCI and mild dementia: Preparatory phase

INTRODUCTION

Etiological diagnosis of neurocognitive disorders of middle-old age relies on biomarkers, although evidence for their rational use is incomplete. A European task force is defining a diagnostic workflow where expert experience fills evidence gaps for biomarker validity and prioritization. We report methodology and preliminary results.

METHODS

Using a Delphi consensus method supported by a systematic literature review, 22 delegates from 11 relevant scientific societies defined workflow assumptions.

RESULTS

We extracted diagnostic accuracy figures from literature on the use of biomarkers in the diagnosis of main forms of neurocognitive disorders. Supported by this evidence, panelists defined clinical setting (specialist outpatient service), application stage (MCI-mild dementia), and detailed pre-assessment screening (clinical-neuropsychological evaluations, brain imaging, and blood tests).

DISCUSSION

The Delphi consensus on these assumptions set the stage for the development of the first pan-European workflow for biomarkers' use in the etiological diagnosis of middle-old age neurocognitive disorders at MCI-mild dementia stages.

HIGHLIGHTS

Rational use of biomarkers in neurocognitive disorders lacks consensus in Europe. A consensus of experts will define a workflow for the rational use of biomarkers. The diagnostic workflow will be patient-centered and based on clinical presentation. The workflow will be updated as new evidence accrues.

Evaluation of Resilience in Historic Urban Areas by Combining Multi-Criteria Decision-Making System and GIS, with Sustainability and Regeneration Approach: The Case Study of Tehran (IRAN)

Historic urban areas are the beating heart of the city, but neglecting them can lead to low resilience. Therefore, paying attention to their regeneration can create a sustainable city. The purpose of this study was to determine the resilience of neighborhoods in Tehran and evaluate effective criteria for the resilience increase. In this study, to evaluate the resilience of Tehran, initially, 18 criteria were considered. Then, using the Delphi technique, 14 criteria among them were selected for final analysis. Using the AHP multi-criteria decision-making method, the importance of each criterion was determined. Using GIS capabilities, the parameters map was prepared, and by combining the prepared maps with AHP weights, a resilience map was created. Finally, 20 neighborhoods with the lowest resilience were identified as priorities for stabilization and regeneration measures, and the criteria status used in them was examined. Results showed that deteriorated urban areas (19.53%) and construction materials (18.51%) were the most important criteria. Non-resilience areas were generally in the southern half of the city. 78% of 20 selected neighborhoods had deteriorated urban areas, while only 14% of the city deteriorated. Finally, by examining the criteria in neighborhoods with the lowest resilience, suggestions were made to regeneration, sustainability, and increase the resilience of these neighborhoods.

How Will We Dine? Prospective Shifts in International Haute Cuisine and Innovation beyond Kitchen and Plate

Haute cuisine, the cooking style for fine dining at gourmet restaurants, has changed over the last decades and can be expected to evolve in the upcoming years. To engage in foresight, the purpose of this study is to identify a plausible future trend scenario for the haute cuisine sector within the next five to ten years, based on today’s chefs’ views. To achieve this goal, an international, two-stage Delphi study was conducted. The derived scenario suggests that the coronavirus disease 2019 (COVID-19) pandemic will lead to significant restaurant bankruptcies and will raise creativity and innovation among the remaining ones. It is expected that haute cuisine tourism will grow and that menu prices will differ for customer segments. More haute cuisine restaurants will open in Asia and America. Local food will remain a major trend and will be complemented by insect as well as plant-based proteins and sophisticated nonalcoholic food pairings. Restaurant design and the use of scents will become more relevant. Also, private dining and fine dining at home will become more important. The scenario also includes negative projections. These findings can serve as a research agenda for future research in haute cuisine, including the extension of the innovation lens towards the restaurant and the business model. Practical implications include the necessity for haute cuisine restaurants to innovate to cope with increasing competition in several regions. Customers should be seen as co-creators of the value of haute cuisine.