The Delphi method and health research

Undergraduate prosthetics and orthotics teaching methods: A baseline for international comparison

BACKGROUND

Education of Prosthetics and Orthotics is a relatively recent professional program. While there has been some work on various teaching methods and strategies in international medical education, limited publication exists within prosthetics and orthotics.

OBJECTIVES

To identify the teaching and learning methods that are used in Bachelor-level prosthetics and orthotics programs that are given highest priority by expert prosthetics and orthotics instructors from regions enjoying a range of economic development.

STUDY DESIGN

Mixed method.

METHODS

The study partly documented by this article utilized a mixed method approach (qualitative and quantitative methods) within which each phase provided data for other phases. It began with analysis of prosthetics and orthotics curricula documents, which was followed by a broad survey of instructors in this field and then a modified Delphi process.

RESULTS

The expert instructors who participated in this study gave high priority to student-centered, small group methods that encourage critical thinking and may lead to lifelong learning. Instructors from more developed nations placed higher priority on student's independent acquisition of prosthetics and orthotics knowledge, particularly in clinical training.

CONCLUSIONS

Application of student-centered approaches to prosthetics and orthotics programs may be preferred by many experts, but there appeared to be regional differences in the priority given to different teaching methods.

CLINICAL RELEVANCE

The results of this study identify the methods of teaching that are preferred by expert prosthetics and orthotics instructors from a variety of regions. This treatment of current instructional techniques may inform instructor choice of teaching methods that impact the quality of education and improve the professional skills of students.

In Search of a Gold Standard Scoring System for the Subjective Evaluation of Cosmetic Outcomes Following Breast-Conserving Therapy

The absence of a widely accepted method for aesthetic evaluation following breast-conserving surgery for breast cancer limits the ability to evaluate cosmetic outcomes. In this study, two different panel scoring approaches were compared in an attempt to identify a gold standard scoring system for subjectively assessing cosmetic outcomes following breast-conserving therapy. Standardized photographs of each participant were evaluated independently by twelve health care professionals involved in breast cancer diagnosis and treatment using the Danoff four-point scale. Individual Danoff scores were combined using two methods, a random sample "three-panel" score and an iterative "Delphi-panel" score, in order to create a final cosmetic score for each patient. Agreement between these two aggregative approaches was assessed with a weighted kappa (wk) statistic. Patient and professional recruitment occurred at two separate tertiary care multi-disciplinary breast health centers. Women with unilateral breast cancer who underwent breast-conserving therapy (segmental mastectomy or lumpectomy and radiotherapy) and were at least 2 years after radiotherapy were asked to participate. Ninety-seven women were evaluated. The Delphi approach required three rounds of evaluation to obtain greater than 50% agreement in all photographs. The wk statistic between scores generated from the "three-panel" and "Delphi-panel" approaches was 0.80 (95% CI: 0.71-0.89), thus demonstrating substantial agreement. Evaluation of cosmetic outcomes following breast-conserving therapy using a "three-panel" and "Delphi-panel" score provide similar results, confirming the reliability of either approach for subjective evaluation. Simplicity of use and interpretation favors the "three-panel" score. Future work should concentrate on the integration of the three-panel score with objective and patient-reported scales to generate a comprehensive cosmetic evaluation platform.

Developing standards for reporting implementation studies of complex interventions (StaRI): a systematic review and e-Delphi

BACKGROUND

Dissemination and implementation of health care interventions are currently hampered by the variable quality of reporting of implementation research. Reporting of other study types has been improved by the introduction of reporting standards (e.g. CONSORT). We are therefore developing guidelines for reporting implementation studies (StaRI).

METHODS

Using established methodology for developing health research reporting guidelines, we systematically reviewed the literature to generate items for a checklist of reporting standards. We then recruited an international, multidisciplinary panel for an e-Delphi consensus-building exercise which comprised an initial open round to revise/suggest a list of potential items for scoring in the subsequent two scoring rounds (scale 1 to 9). Consensus was defined a priori as 80% agreement with the priority scores of 7, 8, or 9.

RESULTS

We identified eight papers from the literature review from which we derived 36 potential items. We recruited 23 experts to the e-Delphi panel. Open round comments resulted in revisions, and 47 items went forward to the scoring rounds. Thirty-five items achieved consensus: 19 achieved 100% agreement. Prioritised items addressed the need to: provide an evidence-based justification for implementation; describe the setting, professional/service requirements, eligible population and intervention in detail; measure process and clinical outcomes at population level (using routine data); report impact on health care resources; describe local adaptations to the implementation strategy and describe barriers/facilitators. Over-arching themes from the free-text comments included balancing the need for detailed descriptions of interventions with publishing constraints, addressing the dual aims of reporting on the process of implementation and effectiveness of the intervention and monitoring fidelity to an intervention whilst encouraging adaptation to suit diverse local contexts.

CONCLUSIONS

We have identified priority items for reporting implementation studies and key issues for further discussion. An international, multidisciplinary workshop, where participants will debate the issues raised, clarify specific items and develop StaRI standards that fit within the suite of EQUATOR reporting guidelines, is planned.

REGISTRATION

The protocol is registered with Equator: http://www.equator-network.org/library/reporting-guidelines-under-development/#17 .

What should we know about dementia in the 21st century? A Delphi consensus study

Background

Escalating numbers of people are experiencing dementia in many countries. With increasing consumer needs, there is anticipated growth in the numbers of people providing diagnostic evaluations, treatments, and care. Ensuring a consistent and contemporary understanding of dementia across all of these groups has become a critical issue. This study aimed to reach consensus among dementia experts from English speaking countries regarding essential and contemporary knowledge about dementia.

Methods

An online Delphi study was conducted to examine expert opinion concerning dementia knowledge with three rounds of data collection. A sample of dementia experts was selected by a panel of Australian experts, including a geriatrician and three professors of aged care. Purposive selection was initially undertaken with the sample expanded through snowballing. Dementia experts (N = 19) included geriatricians, psychologists, psychiatrists, neuroscientists, dementia advocates, and nurse academics from the United Kingdom, United States, and Australia. In the first round, these participants provided open-ended responses to questions determining what comprised essential knowledge about dementia. In the second round, responses were summarised into 66 discrete statements that participants rated on the basis of importance. In the third round, a rank-ordered list of the 66 statements and a group median were provided and participants rated the statements again. The degree of consensus regarding importance ratings was determined by assessing median, interquartile range, and proportion of experts scoring above predetermined thresholds. Correlation scores were calculated for each statement after the final round to identify changes in statement scores.

Results

The Delphi experts identified 36 statements about dementia that they considered essential to understanding the condition. Statements about care for a person experiencing dementia and their care giver represented the largest response category. Other statements, for which full or very high consensus was reached, related to dementia characteristics, symptoms and progression, diagnosis and assessment, and treatment and prevention.

Conclusions

These results summarise knowledge of dementia that is considered essential across expert representatives of key stakeholder groups from three countries. This information has implications for the delivery of care to people with the condition and the development of dementia education programs.

A Delphi study of pharmacotherapy for noncystic fibrosis bronchiectasis

BACKGROUND

Presently, there are no medications approved by regulatory bodies for the treatment of noncystic fibrosis bronchiectasis (NCFB). In addition, most techniques/devices to enhance mucociliary clearance in NCFB have not been rigorously evaluated. Therefore, we performed a Delphi study of clinical experts concerning the treatment of NCFB.

METHODS

Ten individuals with extensive experience with NCFB participated. Questions were posed concerning pharmacotherapy and mucocilary clearance techniques for NCFB. Iterative questioning and feedback from previous responses were used with an aim to reach consensus. Consensus was defined as ≥70% agreement of the experts.

RESULTS

The experts reached consensus concerning several issues including: (1) 10 to 14 days duration of antibiotics for exacerbations of NCFB, (2) Combination antibiotics should not be given for acute exacerbations of NCFB regardless of history of pseudomonas colonization, (3) some type of airway clearance technique should be used for stable NCFB, (4) recommended treatment end points for NCFB include sputum volume, sputum color and exacerbation frequency and (5) recombinant DNA-ase and inhaled corticosteroids should not be routinely used for NCFB. There was some inconsistency in expert responses between the 2 rounds of questions, suggesting that even NCFB experts do not have a completely consistent approach to treatment.

CONCLUSIONS

NCFB experts reached consensus on several treatment issues. In the absence of rigorous medical evidence, these data supply support for these approaches. Several questions posed to the experts did not reach consensus. We would classify these issues as "controversial" and suggest that they are important areas for future research.

Instrument for scoring clinical outcome of research for epidermolysis bullosa: a consensus-generated clinical research tool

Epidermolysis bullosa (EB) is a genetic condition characterized by skin fragility and blistering. There is no instrument available for clinical outcome research measurements. Our aim was to develop a comprehensive instrument that is easy to use in the context of interventional studies. Item collection was accomplished using a two-step Delphi Internet survey process for practitioners and qualitative content analysis of patient and family interviews. Items were reduced based on frequency and importance using a 4-point Likert scale and were subject to consensus (>80% agreement) using the nominal group technique. Pilot data testing was performed in 21 consecutive patients attending an EB clinic. The final score, Instrument for Scoring Clinical Outcome of Research for Epidermolysis Bullosa (iscorEB), is a combined score that contains clinician items grouped in five domains (skin, mucosa, organ involvement, laboratory abnormalities, and complications and procedures; maximum score 114) and patient-derived items (pain, itch, functional limitations, sleep, mood, and effect on daily and leisurely activities; maximum score 120). Pilot testing revealed that combined (see below) and subscores were able to differentiate between EB subtypes and degrees of clinical severity (EB simplex 21.7 ± 16.5, junctional EB 28.0 ± 20.7, dystrophic EB 57.3 ± 24.6, p = 0.007; mild 17.3 ± 9.6, moderate 41.0 ± 19.4, and severe 64.5 ± 22.6, p < 0.001). There was high correlation between clinician and patient subscores (correlation coefficient = 0.79, p < 0.001). iscorEB seems to be a sensitive tool in differentiating between EB types and across the clinical spectrum of severity. Further validation studies are needed.

Patterns of change in implementation of state alcohol control policies in the United States, 1999-2011

AIMS

To examine state alcohol control policy implementation by policy efficacy and intent.

DESIGN

A descriptive longitudinal analysis of policy implementation.

SETTING

The United States, 1999-2011.

PARTICIPANTS

Fifty states and the District of Columbia.

MEASUREMENTS

Twenty-nine state-level policies were rated based on an implementation rating (IR; range = 0.0-1.0) gathered from the Alcohol Policy Information System, government and industry reports and other sources; and expert judgment about policy efficacy for addressing binge drinking and alcohol-impaired driving among the general population and youth, respectively.

FINDINGS

On average, implementation of the most effective general population policies did not change [mean IR = 0.366 in 1999; 0.375 in 2011; slope for annual change = 0.001; 95% confidence interval (CI) for the slope -0.001, 0.002]. In contrast, implementation increased over time for less effective policies (mean IR = 0.287 in 1999; 0.427 in 2011; slope for annual change compared with most effective policies = 0.009; slope 95% CI = 0.002-0.007), for youth-oriented policies (mean IR = 0.424 in 1999; 0.511 in 2011; slope for annual change compared with most effective policies = 0.007; slope 95% CI = 0.005-0.009), and for impaired driving policies (mean IR = 0.493 in 1999; 0.608 in 2011; slope for annual change compared with most effective policies = 0.0105; slope 95% CI = 0.007-0.014).

CONCLUSIONS

Implementation of politically palatable state alcohol policies, such as those targeting youth and alcohol-impaired driving, and less effective policies increased during 1999-2011 in the United States, while the most effective policies that may maximally protect public health remained underused.

A consensus on men's health status and policy in Asia: a Delphi survey

OBJECTIVE

There is currently no documentation on the availability and implementation of policies related to men's health in Asia. This Delphi study aimed to achieve an Asian consensus on men's health policy based on the opinions and recommendations from men's health key opinion leaders.

STUDY DESIGN

A two-phase Delphi online survey was used to gather information from men's health stakeholders across Asian countries.

METHODS

All stakeholders were invited to participate in the survey through men's health conferences, personal contacts, recommendations from international men's health organizations and snowballing method. Stakeholders were asked about their concerns on 17 men's health key issues as well as their opinion on the availability and recommendations on men's health policies and programmes in their countries.

RESULTS

There were a total of 128 stakeholders (policy makers, clinicians, researchers and consumers), from 28 Asian countries, who responded in the survey. Up to 85% of stakeholders were concerned about various men's health issues in Asia and in their respective country, particularly in smoking, ischaemic heart disease and high blood pressure. There is a lack of men's health policies and programmes in Asia (availability = 11.6-43.5%) and up to 92.9% of stakeholders recommended that these should be developed.

CONCLUSIONS

These findings call for policy change and development, and more importantly a concerted effort to elevate men's health status in Asia.

Horizon 2020 priorities in clinical mental health research: results of a consensus-based ROAMER expert survey

Within the ROAMER project, which aims to provide a Roadmap for Mental Health Research in Europe, a two-stage Delphi survey among 86 European experts was conducted in order to identify research priorities in clinical mental health research. Expert consensus existed with regard to the importance of three challenges in the field of clinical mental health research: (1) the development of new, safe and effective interventions for mental disorders; (2) understanding the mechanisms of disease in order to be able to develop such new interventions; and (3) defining outcomes (an improved set of outcomes, including alternative outcomes) to use for clinical mental health research evaluation. Proposed actions involved increasing the utilization of tailored approaches (personalized medicine), developing blended eHealth/mHealth decision aids/guidance tools that help the clinician to choose between various treatment modalities, developing specific treatments in order to better target comorbidity and (further) development of biological, psychological and psychopharmacological interventions. The experts indicated that addressing these priorities will result in increased efficacy and impact across Europe; with a high probability of success, given that Europe has important strengths, such as skilled academics and a long research history. Finally, the experts stressed the importance of creating funding and coordinated networking as essential action needed in order to target the variety of challenges in clinical mental health research.

Disentangling rhetoric and reality: an international Delphi study of factors and processes that facilitate the successful implementation of decisions to decommission healthcare services

Background

The need to better understand processes of removing, reducing, or replacing healthcare services that are no longer deemed essential or effective is common across publicly funded healthcare systems. This paper explores expert international opinion regarding, first, the factors and processes that shape the successful implementation of decommissioning decisions and, second, consensus as to current best practice.

Methods

A three round Delphi study of 30 international experts was undertaken. In round one, participants identified factors that shape the outcome of decommissioning processes; responses were analysed using conventional content analysis. In round two, responses to 88 Likert scale statements derived from round one were analysed using measures of the degree of consensus. In round three the statements that achieved low consensus were then repeated but presented alongside the overall results from round two. The responses were re-analysed to observe whether the degree of consensus had changed. Any open comments provided during the Delphi study were analysed thematically.

Results

Participants strongly agreed that three considerations should ideally inform decommissioning decisions: quality and patient safety, clinical effectiveness and cost-effectiveness. Although there was less consensus as to which considerations informed such decisions in practice, those that drew the most agreement were: cost/budgetary pressures, government intervention and capital costs/condition. Important factors in shaping decommissioning were: strength of executive leadership, strength of clinical leadership, quality of communications, demonstrable benefits and clarity of rationale/case for change. Amongst the 19 best practice recommendations high consensus was achieved for: establishing a strong leadership team, engaging clinical leaders from an early stage, and establishing a clear rationale for change.

Conclusions

There was a stark contrast between what experts thought should determine decommissioning decisions and what does so in practice; a contrast mirrored in the distinction the participants drew between the technical and political aspects of decommissioning processes. The best practice recommendations which we grouped into three categories—change management and implementation; evidence and information; and relationships and political dimensions—can be seen as contemporary responses or strategies to manage the tensions that emerged between the rhetoric and reality of implementing decommissioning decisions.

Determining the Role of a National Objective Assessment of Surgical Skills in Gynecological Oncology: An e-Delphi Methodology

Objective

The aim of this study is to determine a national consensus on the role of an objective assessment of technical surgical skills in gynecological oncology (GO).

Methods

After approval was obtained from Society of Gynecologic Oncology of Canada, A panel of 20 GO leaders was assembled, representing all GO fellowship programs, and was asked to participate in an anonymous group and respond to an online 49-item questionnaire using a modified Delphi methodology.

Results

Ninety-five percent (n = 19) of those invited to participate did so. Seventeen of the panelists (89.5%) believed there was no sufficiently standardized technical skills assessment for GO fellows, whereas 18 responders (95%) believed that fellows should be objectively assessed on more than 1 occasion during their training. Consensus was predefined as Cronbach α greater than 0.8. The panel agreed on what procedures should be objectively assessed with a Cronbach α of 0.967. An overall Cronbach α of 0.993 was achieved after a single Delphi round.

Conclusions

We achieved consensus on the possible components and logistics of a skills assessment process among a group of highly experienced GO trainers in Canada. This study provides the basis for further investigation and debate on the potential value, necessity, and feasibility of an assessment of advanced surgical and nonsurgical skills of GO trainees.

Framework of behavioral indicators for outcome evaluation of TB health promotion: a Delphi study of TB suspects and Tb patients

Background

Health promotion for prevention and control of Tuberculosis (TB) is implemented worldwide because of its importance, but few reports have evaluated its impact on behavior due to a lack of standard outcome indicators. The objective of this study was to establish a framework of behavioral indicators for outcome evaluation of TB health promotion among TB suspects and patients.

Methods

A two-round modified Delphi method involving sixteen TB control experts was used to establish a framework of behavioral indicators for outcome evaluation of TB health promotion targeted at TB suspects and patients.

Results

Sixteen of seventeen invited experts in TB control (authority score of 0.91 on a 1.0 scale) participated in round 1 survey. All sixteen experts also participated in a second round survey. After two rounds of surveys and several iterations among the experts, there was consensus on a framework of indicators for measuring outcomes of TB health promotion for TB suspects and patients. For TB suspects, the experts reached consensus on 2 domains (“Healthcare seeking behavior” and “Transmission prevention”), 3 subdomains (“Seeking care after onset of TB symptoms”, “Pathways of seeking care” and “Interpersonal contact etiquette”), and 8 indicators (including among others, “Length of patient delay”). For TB patients, consensus was reached on 3 domains (“Adherence to treatment”, “Healthy lifestyle” and “Transmission prevention”), 8 subdomains (including among others, “Adherence to their medication”), and 14 indicators (including “Percentage of patients who adhered to their medication”). Operational definitions and data sources were provided for each indicator.

Conclusions

The findings of this study provide the basis for debate among international experts on a framework for achieving global consensus on outcome indicators for TB health promotion interventions targeted at TB patients and suspects. Such consensus will help to increase effectiveness of TB health promotion, while ensuring international comparability of outcome data.

Community assessment workshops: a group method for gathering client experiences of health services

Community assessment workshops were developed to gather client experiences of primary health care services in Australia. Primary health care services are particularly concerned with working with disadvantaged populations, for whom traditional client survey methods such as written surveys may not be inclusive and accessible. Service staff at six Australian primary health care services, including two Aboriginal-specific services, invited participants to attend workshops in 2011-2012. Participants were offered transport, childcare and an interpreter, and provided with reimbursement for their time. Ten workshops were run with a total of 65 participants who accessed a variety of services and programmes. A mix of age and gender was achieved. The workshops yielded detailed qualitative data and quantitative rankings for nine service qualities: holistic, effective, efficient, culturally respectful, used by those most in need, responsive to the local community, increasing individual control, supports and empowers the community, and mix of treatment, prevention and promotion. Discussions were audio recorded and transcribed for qualitative analysis. The workshop approach succeeded in being (i) inclusive, reaching users from disadvantaged sections of the community; (ii) comprehensive, providing ratings and discussion that took account of the whole service; (iii) richly descriptive, with researchers able to generate detailed feedback; and (iv) more empowering than traditional client survey methods, by allowing more control to participants and greater benefits than surveys of individuals. The community assessment workshops are a method that could be widely applied to health service evaluation research where the goal is to reach disadvantaged communities and provide ratings and detailed analysis of the experience of users. The participants and the research benefited from the group approach, and the workshops provided valuable, actionable information to the health services. Recruitment of users, particularly those from culturally diverse backgrounds, remains one of the key challenges facing evaluators.

Domains of quality of life: results of a three-stage Delphi consensus procedure among patients, family of patients, clinicians, scientists and the general public

Purpose

Our key objective is to identify the core domains of health-related quality of life (QoL). Health-related QoL utility scales are commonly used in economic evaluations to assess the effectiveness of health-care interventions. However, health-care interventions are likely to affect QoL in a broader sense than is quantifiable with traditional scales. Therefore, measures need to go beyond these scales. Unfortunately, there is no consensus in the scientific literature on the essential domains of QoL.

Methods

We conducted a three-stage online Delphi consensus procedure to identify the key domains of health-related QoL. Five stakeholder groups (i.e., patients, family of patients, clinicians, scientists and general public) were asked, on three consecutive occasions, what they perceive as the most important domains of health-related QoL. An analysis of existing (health-related) QoL and well-being measurements formed the basis of the Delphi-procedure.

Results

In total, 42 domains of QoL were judged, covering physical, mental and social aspects. All participants rated ‘self-acceptance’, ‘self-esteem’ and ‘good social contacts’ as essential. Strikingly, mental and social domains are perceived as more essential than physical domains across stakeholders groups.

Conclusions

In traditionally used health-related QoL utility measures, physical domains like ‘mobility’ are prominently present. The Delphi-procedure shows that health-related QoL (utility) scales need to put sufficient emphasis on mental and social domains to capture aspects of QoL that are essential to people.

Global research priorities to better understand the burden of iatrogenic harm in primary care: an international Delphi exercise

Using a modified Delphi exercise, Aziz Sheikh and colleagues identify research priorities for patient safety research in primary care contexts. Please see later in the article for the Editors' Summary.

The desired competence of the Swedish ambulance nurse according to the professionals - a Delphi study

Nursing is evolving into new fields of health care including ambulance care, where a branch of specialist nursing is growing. Various views exist on the desired competence for the ambulance nurse and valid guidelines are lacking in Sweden. To increase knowledge of the field, professionals were asked to describe what competences an ambulance nurse should possess. The aim of this study was therefore to elucidate the desired professional competence of the specialist ambulance nurse, according to the professionals. A modified Delphi technique was used, where a panel of professional experts expressed their views on the desired competence of the ambulance nurse. This study reports, at a high level of agreement among the panel experts, that the desired competence of the specialist ambulance nurse consist of forty-four separate competences creating ten areas of competences: execute leadership, generic abilities, interpersonal communication, institutional collaboration, pedagogic skills, possession of relevant knowledge, professional judgement, professional skills, research activities, and technical skills. The high level of agreement among the professionals as well as the large number of competences reflects the high demands placed on the ambulance nurse by the professionals themselves.

Efficacy and the strength of evidence of U.S. alcohol control policies

BACKGROUND

Public policy can limit alcohol consumption and its associated harm, but no direct comparison of the relative efficacy of alcohol control policies exists for the U.S.

PURPOSE

To identify alcohol control policies and develop quantitative ratings of their efficacy and strength of evidence.

METHODS

In 2010, a Delphi panel of ten U.S. alcohol policy experts identified and rated the efficacy of alcohol control policies for reducing binge drinking and alcohol-impaired driving among both the general population and youth, and the strength of evidence informing the efficacy of each policy. The policies were nominated on the basis of scientific evidence and potential for public health impact. Analysis was conducted in 2010-2012.

RESULTS

Panelists identified and rated 47 policies. Policies limiting price received the highest ratings, with alcohol taxes receiving the highest ratings for all four outcomes. Highly rated policies for reducing binge drinking and alcohol-impaired driving in the general population also were rated highly among youth, although several policies were rated more highly for youth compared with the general population. Policy efficacy ratings for the general population and youth were positively correlated for reducing both binge drinking (r=0.50) and alcohol-impaired driving (r=0.45). The correlation between efficacy ratings for reducing binge drinking and alcohol-impaired driving was strong for the general population (r=0.88) and for youth (r=0.85). Efficacy ratings were positively correlated with strength-of-evidence ratings.

CONCLUSIONS

Comparative policy ratings can help characterize the alcohol policy environment, inform policy discussions, and identify future research needs.

Perceived skills and abilities required by athletic trainers in hospital and clinical management positions: a Delphi study

CONTEXT

Athletic training has expanded from traditional sport-team settings to varied settings involving active populations. Athletic trainers also use their education and abilities in administration to take on roles of management in hospitals and health care clinics.

OBJECTIVE

To begin to explore the knowledge, skills, and abilities needed in the emerging practice setting of health care management.

DESIGN

Delphi study.

SETTING

Directed surveys.

PATIENTS OR OTHER PARTICIPANTS

Eight athletic trainers working as hospital and health care clinic managers in varied geographic settings.

DATA COLLECTION AND ANALYSIS

Three rounds of directed surveys were used and included (1) a series of demographic questions and 1 focused, open-ended question, (2) 32 statements scored on a 6-point Likert-type scale with no neutral statement, and (3) 10 statements ranked in order of importance for the athletic trainer working as a health care manager.

RESULTS

I grouped the results into 2 categories: leadership skills and management tools.

CONCLUSIONS

According to participants, effective health care managers need a strong understanding of business and management tools along with more interpersonal skills in communication and leadership. The results are consistent with the literature and may be applied in athletic training education programs and by athletic trainers seeking health care management positions.

Potential and requirements of IT for ambient assisted living technologies. Results of a Delphi study

OBJECTIVES

Ambient Assisted Living (AAL) technologies are developed to enable elderly to live independently and safely. Innovative information technology (IT) can interconnect personal devices and offer suitable user interfaces. Often dedicated solutions are developed for particular projects. The aim of our research was to identify major IT challenges for AAL to enable generic and sustainable solutions.

METHODS

Delphi Survey. An online questionnaire was sent to 1800 members of the German Innovation Partnership AAL. The first round was qualitative to collect statements. Statements were reduced to items by qualitative content analysis. Items were assessed in the following two rounds by a 5-point Likert-scale. Quantitative analyses for second and third round: descriptive statistics, factor analysis and ANOVA.

RESULTS

RESPONDENTS

81 in first, 173 in second and 70 in third round. All items got a rather high assessment. Medical issues were rated as having a very high potential. Items related to user-friendliness were regarded as most important requirements. Common requirements to all AAL-solutions are reliability, robustness, availability, data security, data privacy, legal issues, ethical requirements, easy configuration. The complete list of requirements can be used as framework for customizing future AAL projects.

CONCLUSIONS

A wide variety of IT issues have been assessed important for AAL. The extensive list of requirements makes obvious that it is not efficient to develop dedicated solutions for individual projects but to provide generic methods and reusable components. Experiences and results from medical informatics research can be used to advance AAL solutions (e.g. eHealth and knowledge-based approaches).

Internet-based Delphi research: case based discussion

The interactive capacity of the Internet offers benefits that are intimately linked with contemporary research innovation in the natural resource and environmental studies domains. However, e-research methodologies, such as the e-Delphi technique, have yet to undergo critical review. This study advances methodological discourse on the e-Delphi technique by critically assessing an e-Delphi case study. The analysis suggests that the benefits of using e-Delphi are noteworthy but the authors acknowledge that researchers are likely to face challenges that could potentially compromise research validity and reliability. To ensure that these issues are sufficiently considered when planning and designing an e-Delphi, important facets of the technique are discussed and recommendations are offered to help the environmental researcher avoid potential pitfalls associated with coordinating e-Delphi research.

New firms … old assets

Purpose

The purpose of this paper is to demonstrate, through the Delphi technique, how demand drivers and accommodation priorities for emerging knowledge‐intensive firms are understood and how corporate property and asset managers can respond to them.

Design/methodology/approach

This empirical paper discusses the results emerging from a Delphi study in four rounds with ten experts. This research focuses on the high growth, commercial sector in south‐east Queensland in Australia.

Findings

The accommodation priority demands of knowledge‐intensive firms varied depending on the stage of their lifecycle, that is, start‐up, established or mature. For start‐ups, accommodation cost was of primary importance; however, for the established and mature firms, their priorities were directly related to those supporting the progress of the firms itself and the productivity of its employers.

Practical implications

Many of the findings are likely to be applicable to commercial environments elsewhere and would provide a basis for investors, developers, asset owners and managers to better align their “old” assets, through better asset management, to these “new” firms within this rapidly changing environment.

Originality/value

The use of the Delphi technique in this multi‐faceted research topic captures the practical knowledge, wisdom and intuition of experts who deal with such issues on a day‐to‐day basis. Most real estate issues usually involve diverse specialities and perspectives – this paper illustrates how common ground through consensus can be achieved.

Undergraduate prosthetics and orthotics programme objectives:a baseline for international comparison and curricular development

BACKGROUND

Prosthetics and orthotics is a relatively recent addition to the suite of undergraduate professional preparation programmes. There has been limited publication regarding international patterns of curriculum development, particularly concerning how objectives differ across global regions.

OBJECTIVES

This paper compares current prosthetics and orthotics curricula from a range of regions and identifies both common and distinctive objectives.

STUDY DESIGN

Mixed method: document analysis followed by modified Delphi process.

METHODS

Documents were analysed qualitatively to compare various curricula and emergent features were evaluated by a group of expert prosthetics and orthotics instructors.

RESULTS

There was substantial agreement that programmes should improve student knowledge and understanding. They should establish and extend student fabrication, communication skills and professional co-operation. However, there appeared to be regional differences in the priority given to critical thinking and clinical reasoning; integration of theory and practice and particular approaches to teaching prosthetics and orthotics.

CONCLUSIONS

This study revealed substantial consensus regarding the importance of clear programme objectives dealing with student abilities, professional skills and contemporary understanding. However, this study also revealed regional differences that may well reward further investigation.

Use of a computer-mediated Delphi process to validate a mass casualty conceptual model

Since the original work on the Delphi technique, multiple versions have been developed and used in research and industry; however, very little empirical research has been conducted that evaluates the efficacy of using online computer, Internet, and e-mail applications to facilitate a Delphi method that can be used to validate theoretical models. The purpose of this research was to develop computer, Internet, and e-mail applications to facilitate a modified Delphi technique through which experts provide validation for a proposed conceptual model that describes the information needs for a mass-casualty continuum of care. Extant literature and existing theoretical models provided the basis for model development. Two rounds of the Delphi process were needed to satisfy the criteria for consensus and/or stability related to the constructs, relationships, and indicators in the model. The majority of experts rated the online processes favorably (mean of 6.1 on a seven-point scale). Using online Internet and computer applications to facilitate a modified Delphi process offers much promise for future research involving model building or validation. The online Delphi process provided an effective methodology for identifying and describing the complex series of events and contextual factors that influence the way we respond to disasters.

A modified Delphi methodology to conduct a failure modes effects analysis: a patient-centric effort in a clinical medical laboratory

In this article, we describe the use of an information-gathering tool, the Delphi technique, to overcome issues encountered when conducting a failure modes effects analysis as part of a define, measure, analyze, implement, control study to improve the processes of a clinical medical laboratory. The study was conducted with the goals of reducing medical errors in the total testing process (TTP) in order to improve patient safety, patient satisfaction, and improve the overall quality of the health care services provided by the subject hospital while meeting its Joint Commission (JC) accreditation requirements. The study found that the Delphi technique was very useful in overcoming 4 barriers encountered in conducting a failure modes effects analysis in a hospital's clinical medical laboratory and in achieving those goals.

Factors underlying smoking relapse prevention: results of an international Delphi study

No definitive picture of the factors determining smoking relapse exists, and many smoking relapse prevention programmes have only modest behavioural effects. This study aims to identify the level of consensus among and compare the opinions of two groups of experts (researchers and coaches who provide smoking cessation courses) regarding factors already studied in relation to smoking relapse, factors that have not yet been addressed and ideas on how to improve prevention programmes. A three-round Delphi method was employed. In the first round, 15 researchers completed an electronic questionnaire on factors associated with relapse. The results were used to develop a structured questionnaire for the second round, which was completed by 47 researchers and 61 coaches. The second-round results were then presented to the same experts in the third round, enabling them to re-rate their answers. Results revealed high consensus on some factors already identified as predicting relapse (e.g. self-efficacy), new factors (e.g. action planning) and several methods to improve prevention programmes. Generally speaking, the researchers and coaches provided similar suggestions. The results paint a picture of the current state of knowledge on relapse-related factors and point the way to areas for further research.

Global Public Health and the United Arab Emirates

The United Arab Emirates (UAE), as a rapidly developing country in the Middle East, is in a good position to contribute to global health. It can share its experience of dealing with public health problems characteristic of a desert environment, and it can learn from the experience of other nations in tackling challenges posed by globalization. The health issues particular to the region include heat stress, the effects of consanguineous marriages on families and society, and exposure to occupational and environmental hazards. As in many other developed countries, the UAE also has public health problems associated with smoking, road traffic accidents, and obesity. The experience of other countries in dealing with infectious diseases has helped the UAE implement preventive measures to cope with infections such as the recent H1N1 pandemic. International collaboration has advantages to the UAE in managing problems pertaining to global public health.

Engaging communication experts in a Delphi process to identify patient behaviors that could enhance communication in medical encounters

Background

The communication literature currently focuses primarily on improving physicians' verbal and non-verbal behaviors during the medical interview. The Four Habits Model is a teaching and research framework for physician communication that is based on evidence linking specific communication behaviors with processes and outcomes of care. The Model conceptualizes basic communication tasks as "Habits" and describes the sequence of physician communication behaviors during the clinical encounter associated with improved outcomes. Using the Four Habits Model as a starting point, we asked communication experts to identify the verbal communication behaviors of patients that are important in outpatient encounters.

Methods

We conducted a 4-round Delphi process with 17 international experts in communication research, medical education, and health care delivery. All rounds were conducted via the internet. In round 1, experts reviewed a list of proposed patient verbal communication behaviors within the Four Habits Model framework. The proposed patient verbal communication behaviors were identified based on a review of the communication literature. The experts could: approve the proposed list; add new behaviors; or modify behaviors. In rounds 2, 3, and 4, they rated each behavior for its fit (agree or disagree) with a particular habit. After each round, we calculated the percent agreement for each behavior and provided these data in the next round. Behaviors receiving more than 70% of experts' votes (either agree or disagree) were considered as achieving consensus.

Results

Of the 14 originally-proposed patient verbal communication behaviors, the experts modified all but 2, and they added 20 behaviors to the Model in round 1. In round 2, they were presented with 59 behaviors and 14 options to remove specific behaviors for rating. After 3 rounds of rating, the experts retained 22 behaviors. This set included behaviors such as asking questions, expressing preferences, and summarizing information.

Conclusion

The process identified communication tasks and verbal communication behaviors for patients similar to those outlined for physicians in the Four Habits Model. This represents an important step in building a single model that can be applied to teaching patients and physicians the communication skills associated with improved satisfaction and positive outcomes of care.

Determinants of awareness, initiation and maintenance of physical activity among the over-fifties: a Delphi study

To develop effective interventions to stimulate physical activity (PA), insight into its underlying variables is needed. The aim of this study was to obtain an overview of the most relevant determinants of awareness, initiation and maintenance of PA among the over-fifties by means of a three-round Delphi study. In the first round, 17 key-experts outlined possible relevant determinants into an open-ended electronic questionnaire. In the second round, 118 experts completed a structured electronic questionnaire that was based on the first round results, in which they scored each determinant on its relevance. In the third round, experts were asked to re-rate the relevance of each determinant, after feedback was given about the group median relevance score. After three rounds, the experts agreed on 30 relevant determinants of the three phases of PA. When compared with longitudinal studies, the Delphi study pointed out new concepts, such as several post-motivational and social and environmental determinants as possible relevant determinants, suggesting that this method has the potential to trace new and promising determinants. The results further showed that next to similarities, much dissimilarity in relevant determinants of awareness, initiation and maintenance of PA was found, suggesting that most determinants could be phase specific.

Pharmacotherapy for pulmonary sarcoidosis: a Delphi consensus study

BACKGROUND

Most issues concerning pharmacotherapy of pulmonary sarcoidosis have not been resolved in clinical trials. The objective was to survey sarcoidosis experts concerning the treatment of pulmonary sarcoidosis and attempt to reach a consensus by these experts using a Delphi method.

METHODS

A 6-item questionnaire was developed. Experts were identified at the Diffuse Lung Disease Network at the annual CHEST meeting in October 2008. Three rounds of questionnaires were presented to the experts. Respondent feedback and supporting literature was incorporated into the questionnaires of subsequent rounds.

RESULTS

Experts reached a consensus concerning the following issues: (a) corticosteroids are the initial therapy of choice; (b) initial use of inhaled corticosteroids are not recommended; (c) methotrexate was the preferred second-line drug; (d) 40mg of daily prednisone equivalent was the maximum dose recommended for the treatment of acute pulmonary sarcoidosis; (e) tapering to 10mg of daily prednisone equivalent for chronic pulmonary sarcoidosis was considered a successful taper. The experts could not resolve the following issues: (a) the initial corticosteroid dose for the treatment of acute pulmonary sarcoidosis; (b) the decision and timing of corticosteroid therapy in a patient with mild, Stage 2 pulmonary sarcoidosis.

CONCLUSIONS

This Delphi study revealed that sarcoidosis experts reached a consensus concerning several aspects of the treatment of pulmonary sarcoidosis; these could be considered as appropriate approaches to therapy. Other issues concerning the therapy of pulmonary sarcoidosis remain unresolved by experts, and are areas where further clinical research could be directed.

Regional inequality in health and its determinants: evidence from China

Health inequality is a problem with great political importance all over the world. Urban-rural inequality in health has attracted great attentions in recent years in China, but very few researches have been undertaken into regional discrepancies in health. This research aims at measuring the degree of regional health inequality in China and identifying its determinants. Indicators for health, socioeconomic status, health resources and health services delivery were selected through Delphi consultations from 18 experts. With cross-sectional data from 31 provinces, composite health indexes were generated. The regional inequality in health was described by Lorenz curve and measured by Gini coefficient. The determinants of health inequality were identified through canonical correlation analysis. The results showed that there existed distinct regional disparities in health in China, which were mainly reflected in "Maternal & Child Health" and "Infectious Diseases", not in the most commonly used health indicator average life expectancy. The regional health inequality in China was increasing with the rapid economic growth. The regional health inequality was associated with not only the distribution of wealth, but also the distribution of health resources and primary health care services. Policy makers need to be aware of three major challenges when they try to achieve and maintain equality in distribution of health: First, the most commonly used health indicators are not necessarily sensitive enough to detect health inequalities. Second, increase in health inequality is often accompanied with rapid economic growth and increase in life expectancy. Countries in transition are facing the greatest challenge in developing a fair and equitable health care system. Finally, investment in health resources does bring about differences in distribution of health. However, primary health care plays a more important role than hospital services in reducing regional disparities in health.

Reducing referral-to-treatment waiting times in cancer patients using a multidisciplinary database

INTRODUCTION

Modern delivery of cancer care through patient-centred multidisciplinary teams (MDT) has improved survival. This approach, however, requires effective on-going co-ordination between multiple specialties and resources and can present formidable organisational challenges. The aim of this study was to improve the efficiency of the MDT process for head and neck cancer.

PATIENTS AND METHODS

A systems analysis of the MDT process was undertaken to identify bottlenecks delaying treatment planning. The MDT process was then audited. A revised process was developed and an Intranet-based data management solution was designed and implemented. The MDT process was re-evaluated to complete the audit cycle.

RESULTS

We designed and implemented a trust-wide menu-driven database with interfaces for registering and tracking patients, and automated worklists for pathology and radiology. We audited our MDT for 11 and 10 weeks before and following the introduction of the database, with 226 and 187 patients being discussed during each period. The database significantly improved cross-specialtity co-ordination, leading to a highly significant reduction in the number of patients whose treatment planning was delayed due to unavailability of adjunctive investigations (P < 0.001). This improved the overall efficiency of the MDT by 60%.

CONCLUSIONS

The NHS Cancer Plan aspires to reduce the referral-to-treatment time to 1 month. We have shown that a simple, trust-wide database reduces treatment planning delays in a sizeable proportion of head and neck cancer patients with minimal resource implications. This approach could easily be applied in other MDT meetings.

Developing a competence-based core curriculum in biomedical laboratory science: a Delphi study

In this study the Delphi technique has been used to develop a core curriculum for education of the biomedical scientist. The rapid development in biomedicine and the corresponding changes in methodology in biomedical laboratories demand careful planning of the education of biomedical scientists. The Delphi technique uses an anonymous panel of experts for suggestions and assessments aiming at consensus. Twenty-six experts from different kinds of hospital and university laboratories took part in the investigation. They suggested and assessed necessary competences for a recently graduated biomedical scientist, and if 75% or more of the participants agreed on a competence, it was included in the core curriculum. The final list consisted of 66 competences of varying depth, in three categories. This list contained several generic competences, concerning for example basic laboratory methods, handling of samples, dealing with apparatus and applying relevant rules and laws; basic knowledge in chemistry, preclinical medicine and laboratory methods; and finally attitudes that the panel expected in the recently graduated person. The core was sufficiently restricted to be used in a three-year programme and still leave space for about one year of electives/special study modules. It became rather traditional, e.g. it did not include competences that many recent reports consider important for the future professional.