How do stakeholder groups vary in a Delphi technique about primary mental health care and what factors influence their ratings?

Content validation of a new measure of patient-reported barriers to antiretroviral therapy adherence, the I-Score: results from a Delphi study

Background

Over a third of people living with HIV (PLHIV) have suboptimal adherence to antiretroviral therapy (ART). Measures of barriers to ART adherence often lack comprehensiveness. To help manage ART adherence barriers in HIV care, we are developing a new patient-reported outcome measure (PROM) of these barriers (the I-Score).

Methods

We assessed the content validity of 100 items (distinct barriers) to retain only those most relevant to both PLHIV and HIV health/social service providers. A web-based Delphi was conducted in Canada and France, collecting data from December 2018 to October 2019. Items were evaluated on relevance (the combined rated importance and actionability for HIV care of items among both PLHIV and providers); comprehensibility (rated item clarity); comprehensiveness (examined against our conceptual framework); cross-cultural equivalence (based on comparisons by questionnaire language (English, French) and country of residence). Pearson’s chi-square tests were used for comparisons by language, country, gender, and stakeholder group (PLHIV, providers).

Results

Panelists included 40 PLHIV and 57 providers (66% response rate). Thirty-one items were retained based on consensus thresholds for relevance (minimum: 50% for PLHIV, 60% for providers) and showed good comprehensibility and comprehensiveness, when compared to our conceptual framework (representation of: 6/6 domains, 15/20 subdomains). No significant difference in relevance based on language or country was found among retained items, suggestive of cross-cultural equivalence. Among all 100 items, only 6 significant differences on relevance were observed for gender. For 62 items, the relevance ratings of PLHIV and providers differed significantly, with providers showing greater endorsement of all items but one.

Discussion

The Delphi led to a much-needed item reduction. Remaining items highlight the panel’s multidimensional priorities for the PROM on ART adherence barriers, with few, if any, differences by language, country, and gender. While the analyses may lack generalizability and power, the sample size is considered adequate for a PROM validation study.

Conclusion

Retained items showed good content validity. The different patterns of item endorsement observed underscore the utility of engaging multiple stakeholder groups in PROM development for use in clinical practice. The greater endorsement of items by providers versus patients merits further investigation, including the implications of such differentials for measure development.

Supplementary Information

The online version contains supplementary material available at 10.1186/s41687-022-00435-0.

Quality of care for people with multimorbidity: a focus group study with patients and their relatives

BACKGROUND

Prevalence of people with multimorbidity rises. Multimorbidity constitutes a challenge to the healthcare system, and treatment of patients with multimorbidity is prone to high-quality variations. Currently, no set of quality indicators (QIs) exists to assess quality of care, let alone incorporating the patient perspective. We therefore aim to identify aspects of quality of care relevant to the patients' perspective and match them to a literature-based set of QIs.

METHODS

We conducted eight focus groups with patients with multimorbidity and three focus groups with patients' relatives using a semistructured guide. Data were analysed using Kuckartz's qualitative content analysis. We derived deductive categories from the literature, added inductive categories (new quality aspects) and translated them into QI.

RESULTS

We created four new QIs based on the quality aspects relevant to patients/relatives. Two QIs (patient education/self-management, regular updates of medication plans) were consented by an expert panel, while two others were not (periodical check-ups, general practitioner-coordinated care). Half of the literature-based QIs, for example, assessment of biopsychosocial support needs, were supported by participants' accounts, while more technical domains regarding assessment and treatment regimens were not addressed in the focus groups.

CONCLUSION

We show that focus groups with patients and relatives adding relevant aspects in QI development should be incorporated by default in QI development processes and constitute a reasonable addition to traditional QI development. Our QI set constitutes a framework for assessing the quality of care in the German healthcare system. It will facilitate implementation of treatment standards and increase the use of existing guidelines, hereby helping to reduce overuse, underuse and misuse of healthcare resources in the treatment of patients with multimorbidity.

TRIAL REGISTRATION NUMBER

German clinical trials registry (DRKS00015718), Pre-Results.

Enhancing organizational readiness for implementation: constructing a typology of readiness-development strategies using a modified Delphi process

BACKGROUND

Knowledge about the development of organizational readiness for implementation (ORI) is limited. ORI, referred to as the willingness and capacity of all relevant stakeholders to change practice, is critical for increasing the adoption rate of evidence-based practices and improving implementation outcomes. However, no methodology currently guides ORI's enhancement or addresses differences in readiness needs across an organization. This study used the transtheoretical model (TTM) as a framework for classifying a well-established compilation of implementation strategies into three readiness stages: pre-contemplation, contemplation, and preparation.

METHODS

A modified Delphi method was used to establish consensus among a panel of purposefully selected research and field implementation experts. The Delphi process involved three rounds of online questionnaires. The third round also included a live video discussion to clarify definitions in an effort to increase consensus among experts.

RESULTS

Of the 73 strategies reviewed, the experts identified 75% (n = 55) as relevant for pre-implementation and reached a high-level agreement on the assignment of 7% (n = 5) of the strategies to the pre-contemplation stage (ORI-1), 25% (n = 18) to the contemplation stage (ORI-2), and 52% (n = 38) to the preparation stage (ORI-3). Several strategies were identified as relevant to more than one stage.

CONCLUSIONS

Participating experts were able to reach high-level agreement on the relevance of specific sets of implementation strategies to each of the three ORI stages. The lowest number of strategies was assigned to ORI-1 and the highest number to ORI-3. Given the overlap of strategies across ORI stages, there is a need to better understand the specific utilization of such strategies at different stages. Future studies are needed to empirically evaluate the relevance and applicability of this expert-informed typology based on implementers' experiences in the field.

Delphi Technique in Health Sciences: A Map

Objectives: In health sciences, the Delphi technique is primarily used by researchers when the available knowledge is incomplete or subject to uncertainty and other methods that provide higher levels of evidence cannot be used. The aim is to collect expert-based judgments and often to use them to identify consensus. In this map, we provide an overview of the fields of application for Delphi techniques in health sciences in this map and discuss the processes used and the quality of the findings. We use systematic reviews of Delphi techniques for the map, summarize their findings and examine them from a methodological perspective.

Methods: Twelve systematic reviews of Delphi techniques from different sectors of the health sciences were identified and systematically analyzed.

Results: The 12 systematic reviews show, that Delphi studies are typically carried out in two to three rounds with a deliberately selected panel of experts. A large number of modifications to the Delphi technique have now been developed. Significant weaknesses exist in the quality of the reporting.

Conclusion: Based on the results, there is a need for clarification with regard to the methodological approaches of Delphi techniques, also with respect to any modification. Criteria for evaluating the quality of their execution and reporting also appear to be necessary. However, it should be noted that we cannot make any statements about the quality of execution of the Delphi studies but rather our results are exclusively based on the reported findings of the systematic reviews.

What are the main patient safety concerns of healthcare stakeholders: a mixed-method study of Web-based text

•

Due to the importance of safety in quality care, it’s national policy should be created using a bottom-up approach from various healthcare stakeholders.

•

To explore latent concerns of consumers, providers, government bodies, and researchers, text data analysis on patient safety collected from websites was useful for summarizing various aspects of concern.

•

A common concern among stakeholders was hospital infection control, ranging from nosocomial infections to those brought in by visiting patients around the Patient Safety Act legislation of Korea in 2015.

•

Researchers were focused on hospital sociocultural factors at both the organizational and clinician levels. Government policies and systemic approaches to patient safety were highlighted by different stakeholders.

•

Five topics including infection control showed statistically significant increasing trends over time, while another five showed decreasing trends.

Objectives

Various healthcare stakeholders define quality of care in different ways. Public policy could advocate all these concerns. This study was conducted to identify the main themes on patient safety of stakeholders expressed before and after the Patient Safety Act was enacted in Korea in 2015.

Design

Longitudinal observational study of the interests of healthcare stakeholders generated between January 2014 and September 2018.

Materials and methods

Text data were collected from 2,487 documents on 18 websites that were identified as representative healthcare stakeholder groups of consumers, providers, government, and researchers. A Korean natural language processing (NLP) package, manual review, and synonym dictionary were used for data preprocessing, and we adopted the unsupervised NLP method of probabilistic topic modeling and latent Dirichlet allocation. A linear trend analysis over time, a qualitative step involving two external experts, and original text reviews were performed to validate the identified topics.

Results

Forty-one topics were identified, and the most common concerns of stakeholders were institutional infection control as triggered by the Middle East respiratory syndrome outbreak in early 2015, and infusion-related infection from late 2017 until the middle of 2018. The other top-three concerns of the stakeholder groups were highly similar, while research topics were limited to the perceptions of providers and the activities and culture of hospitals. Five topics showed statistically significant increasing trends over time, while another five showed decreasing trends (both P < 0.05). In the qualitative step, we confirmed 35 themes and revised the other 6.

Conclusions

A common concern among stakeholders was hospital infection control, ranging from nosocomial infections to those brought in by family visiting patients. Government policies and systemic approaches to patient safety were highlighted by different stakeholders. Researchers were focused on hospital sociocultural factors at both the organizational and clinician levels. These identified concerns all should be advocated by the public health policy.

Ambulatory care-sensitive emergency department cases: a mixed methods approach to systemize and analyze cases in Germany

BACKGROUND

Internationally, emergency departments (ED) are treating increasing numbers of patients with conditions that could have been managed appropriately in ambulatory care (AC) settings. The aim of our study was to develop the first consensus-based list of AC-sensitive conditions commonly seen in German EDs and explore predictors of these visits.

METHODS

Our study used a Delphi survey of 30 physicians to compile a list of conditions they agreed were amenable to AC treatment. The group identified reasons why patients visit EDs instead of AC. We used the results to inform spatial regression models analysing the association of patient characteristics and attributes of AC with AC-sensitive ED visits based on 2015 district-level data.

RESULTS

Our study provides a list of AC-sensitive conditions based on the German ED context. Results suggest that, up to the age of 70 years, the older the patients, the less likely they seek EDs for these conditions. Results of our regression analyses suggest that AC-sensitive ED rates were significantly higher in districts with lower physician density. Patients' urgency perception and preferences were identified as main drivers of AC-sensitive ED visits.

CONCLUSION

Future policy measures should aim to help guide patients through the healthcare system so that they receive the best care in place that is most appropriate in terms of quality, safety and continuity of care. A list of AC-sensitive ED conditions can be used as a monitoring instrument and for further analyses of routine data to inform policy makers seeking to improve resource use and allocation.

Clinical indicators for common paediatric conditions: Processes, provenance and products of the CareTrack Kids study

BACKGROUND

In order to determine the extent to which care delivered to children is appropriate (in line with evidence-based care and/or clinical practice guidelines (CPGs)) in Australia, we developed a set of clinical indicators for 21 common paediatric medical conditions for use across a range of primary, secondary and tertiary healthcare practice facilities.

METHODS

Clinical indicators were extracted from recommendations found through systematic searches of national and international guidelines, and formatted with explicit criteria for inclusion, exclusion, time frame and setting. Experts reviewed the indicators using a multi-round modified Delphi process and collaborative online wiki to develop consensus on what constituted appropriate care.

RESULTS

From 121 clinical practice guidelines, 1098 recommendations were used to draft 451 proposed appropriateness indicators. In total, 61 experts (n = 24 internal reviewers, n = 37 external reviewers) reviewed these indicators over 40 weeks. A final set of 234 indicators resulted, from which 597 indicator items were derived suitable for medical record audit. Most indicator items were geared towards capturing information about under-use in healthcare (n = 551, 92%) across emergency department (n = 457, 77%), hospital (n = 450, 75%) and general practice (n = 434, 73%) healthcare facilities, and based on consensus level recommendations (n = 451, 76%). The main reason for rejecting indicators was 'feasibility' (likely to be able to be used for determining compliance with 'appropriate care' from medical record audit).

CONCLUSION

A set of indicators was developed for the appropriateness of care for 21 paediatric conditions. We describe the processes (methods), provenance (origins and evolution of indicators) and products (indicator characteristics) of creating clinical indicators within the context of Australian healthcare settings. Developing consensus on clinical appropriateness indicators using a Delphi approach and collaborative online wiki has methodological utility. The final indicator set can be used by clinicians and organisations to measure and reflect on their own practice.

Neurocognitive disorders: what are the prioritized caregiver needs? A consensus obtained by the Delphi method

Background

The symptoms related to neurocognitive disorders (NCD) may lead to caregiver burden increase. Involving caregivers in research may be an effective way of improving the practicalities and relevance of interventions. The aim of this study was to gather opinion and gain consensus on the caregivers ‘priorities, using a Delphi method and including aspects of needs in pharmaceutical dimension.

Methods

Observational study using a modified Delphi method. This study was conducted in the Clinical and Research Memory Center of the University Hospital of Lyon (France), between September 2015 and January 2016. The expert panel was composed of 68 informal caregivers of people with subjective cognitive decline or NCD living at home.

Results

Caregivers assigned a very high importance to the dimension “information needs about their relative’s disease”, i.e. information on the disease, the treatment and the research; and to “coping skills”, i.e. skills related to emotional support, communication, relationship evolution with the relative and skills to cope with behavioural crisis, behavioural and cognitive disorders. The aspect “coping with behavioural disorders” received a high selection rate (83%).

Conclusions

The main needs selected can be used to design relevant interventions and give guidance to policy to support caregivers. To meet caregiver’s needs, interventions should focus on information about disease and treatment and psychoeducational interventions.

Perceptions of health managers and professionals about mental health and primary care integration in Rio de Janeiro: a mixed methods study

BACKGROUND

Community-based primary mental health care is recommended in low and middle-income countries. The Brazilian Health System has been restructuring primary care by expanding its Family Health Strategy. Due to mental health problems, psychosocial vulnerability and accessibility, Matrix Support teams are being set up to broaden the professional scope of primary care. This paper aims to analyse the perceptions of health professionals and managers about the integration of primary care and mental health.

METHOD

In this mixed-method study 18 health managers and 24 professionals were interviewed from different primary and mental health care services in Rio de Janeiro. A semi-structured survey was conducted with 185 closed questions ranging from 1 to 5 and one open-ended question, to evaluate: access, gateway, trust, family focus, primary mental health interventions, mental health records, mental health problems, team collaboration, integration with community resources and primary mental health education. Two comparisons were made: health managers and professionals' (Mann-Whitney non-parametric test) and health managers' perceptions (Kruskall-Wallis non parametric-test) in 4 service designs (General Traditional Outpatients, Mental Health Specialised Outpatients, Psychosocial Community Centre and Family Health Strategy)(SPSS version 17.0). Qualitative data were subjected to Framework Analysis.

RESULTS

Firstly, health managers and professionals' perceptions converged in all components, except the health record system. Secondly, managers' perceptions in traditional services contrasted with managers' perceptions in community-based services in components such as mental health interventions and team collaboration, and converged in gateway, trust, record system and primary mental health education. Qualitative data revealed an acceptance of mental health and primary care integration, but a lack of communication between institutions. The Mixed Method demonstrated that interviewees consider mental health and primary care integration as a requirement of the system, while their perceptions and the model of work produced by the institutional culture are inextricably linked.

CONCLUSION

There is a gap between health managers' and professionals' understanding of community-based primary mental health care. The integration of different processes of work entails both rethinking workforce actions and institutional support to help make changes.

Three nested randomized controlled trials of peer-only or multiple stakeholder group feedback within Delphi surveys during core outcome and information set development

BACKGROUND

Methods for developing a core outcome or information set require involvement of key stakeholders to prioritise many items and achieve agreement as to the core set. The Delphi technique requires participants to rate the importance of items in sequential questionnaires (or rounds) with feedback provided in each subsequent round such that participants are able to consider the views of others. This study examines the impact of receiving feedback from different stakeholder groups, on the subsequent rating of items and the level of agreement between stakeholders.

METHODS

Randomized controlled trials were nested within the development of three core sets each including a Delphi process with two rounds of questionnaires, completed by patients and health professionals. Participants rated items from 1 (not essential) to 9 (absolutely essential). For round 2, participants were randomized to receive feedback from their peer stakeholder group only (peer) or both stakeholder groups separately (multiple). Decisions as to which items to retain following each round were determined by pre-specified criteria.

RESULTS

Whilst type of feedback did not impact on the percentage of items for which a participant subsequently changed their rating, or the magnitude of change, it did impact on items retained at the end of round 2. Each core set contained discordant items retained by one feedback group but not the other (3-22 % discordant items). Consensus between patients and professionals in items to retain was greater amongst those receiving multiple group feedback in each core set (65-82 % agreement for peer-only feedback versus 74-94 % for multiple feedback). In addition, differences in round 2 scores were smaller between stakeholder groups receiving multiple feedback than between those receiving peer group feedback only. Variability in item scores across stakeholders was reduced following any feedback but this reduction was consistently greater amongst the multiple feedback group.

CONCLUSIONS

In the development of a core outcome or information set, providing feedback within Delphi questionnaires from all stakeholder groups separately may influence the final core set and improve consensus between the groups. Further work is needed to better understand how participants rate and re-rate items within a Delphi process.

TRIAL REGISTRATION

The three randomized controlled trials reported here were each nested within the development of a core information or outcome set to investigate processes in core outcome and information set development. Outcomes were not health-related and therefore trial registration was not applicable.

Agreeing the content of a patient-reported outcome measure for primary care: a Delphi consensus study

BACKGROUND

As the first contact for any health-related need, primary care clinicians often address multiple patient problems, with a range of possible outcomes. There is currently no patient-reported outcome measure (PROM) which covers this range of outcomes. Therefore, many research studies into primary care services use PROMs that do not capture the full impact of these services.

OBJECTIVE

The study aim was to identify outcomes sought by primary care patients which clinicians can influence, thus providing the basis for a new primary care PROM.

METHODS

We used a Delphi process starting with an outcomes list inductively derived in a prior qualitative study. Thirty-five experts were recruited into patient, clinician and academic panels. Participants rated each outcome on whether it was (i) relevant to health, (ii) influenced by primary care and (iii) detectable by patients. In each round, outcomes which passed/failed preset levels of agreement were accepted/rejected. Remaining outcomes continued to the next round.

RESULTS

The process resulted in a set of outcomes occupying the domains of health status, health empowerment (internal and external), and health perceptions. Twenty-six of 36 outcomes were accepted for inclusion in a PROM. Primary care having insufficient influence was the main reason for exclusion.

CONCLUSIONS

To our knowledge, this is the first time PROM outcomes have been agreed through criteria which explicitly exclude outcomes less relevant to health, uninfluenced by primary care or undetected by patients. The PROM in development covers a unique set of outcomes and offers an opportunity for enhanced research into primary care.

Initial development of patient-reported instrument assessing harm, efficacy, and misuse of long-term opioid therapy

Guidelines on long-term opioid therapy recommend frequent reassessment of harm, efficacy, and misuse of these potentially harmful and sometimes ineffective medications. In primary care, there is a need for a brief, patient-reported instrument. This report details the initial steps in the development of such an instrument. An interdisciplinary team of clinician-scientists performed four discrete steps in this study: (1) conceptualization of the purpose and function of the instrument, (2) assembly of an item pool, (3) expert rating on which items were most important to include in the instrument, and (4) modification of expert-selected items based on a reading level check and cognitive interviews with patients. A diverse panel of 47 subject matter experts was presented with 69 items to rate on a 1-9 scale in terms of importance for inclusion in the instrument. The panel highly rated 37 items: 8 related to harm, 4 related to efficacy, and 25 related to misuse. These 37 items were then tested for patient comprehension and modified as needed. Next steps in development will include further item reduction, testing against a gold standard, and assessment of the instrument's effect on clinical outcomes.

The professional perspective on patient involvement in the development of quality indicators: a qualitative analysis using the example of chronic heart failure in the German health care setting

PURPOSE

There is an international consensus that quality indicators (QIs) of health care ought to represent patient-relevant aspects. Therefore, patient involvement in the development process is essential. However, there is no methodological gold standard for involving patients in QI development. The aim of this study is to explore experts' views on the representation of patient-relevant aspects in the QI development process using the QIs developed in the context of the German National Disease Management Guideline for Heart Failure as an example.

METHODS

Semi-structured, open telephone interviews were conducted with 15 German experts (patient representatives, physicians, researchers, and methodologists involved in guideline development or quality assessment). Interview themes were the relevance of the exemplary set of QIs for patients, as well as the legitimacy of, competence of, and collaboration with the patient representative who participated in the development process. Interviews were fully transcribed and content analyzed. Deductive categories derived from the research questions were supplemented by inductively formed categories during the review of the interview material.

RESULTS

The qualitative analysis suggests a discrepancy between the guidelines' QIs and those relevant to patients from an expert's point of view, such as physician-patient communication and quality of counseling. Experts reported only minor communication and cooperation problems while working together in the guideline/QI development team. Concerns existed, for example, regarding the recruitment of patient representatives for diseases without self-help organizations, the financing of patient representation, and the training of patient representatives. Only few potential strategies for improving the process of patient involvement were mentioned.

CONCLUSION

Integrating the patients' perspectives through the recruitment of a patient representative to participate in the development team was well established and broadly accepted. However, experts stated that the finally selected QIs represent only a small part of the patient-relevant aspects of medical care. According to the experts' perceptions, the current processes provide a very limited scope for integrating the patients' perspectives in a more extensive way. Supplementing the set of "conventional" QIs with additional, separately developed, "patient-side" QIs might help to include patient priorities in quality measurement.

User-generated quality standards for youth mental health in primary care: a participatory research design using mixed methods

OBJECTIVES

To develop user-generated quality standards for young people with mental health problems in primary care using a participatory research model.

METHODS

50 young people aged 16-25 from community settings and primary care participated in focus groups and interviews about their views and experiences of seeking help for mental health problems in primary care, cofacilitated by young service users and repeated to ensure respondent validation. A second group of young people also aged 16-25 who had sought help for any mental health problem from primary care or secondary care within the last 5 years were trained as focus groups cofacilitators (n=12) developed the quality standards from the qualitative data and participated in four nominal groups (n=28).

RESULTS

46 quality standards were developed and ranked by young service users. Agreement was defined as 100% of scores within a two-point region. Group consensus existed for 16 quality standards representing the following aspects of primary care: better advertising and information (three); improved competence through mental health training and skill mix within the practice (two); alternatives to medication (three); improved referral protocol (three); and specific questions and reassurances (five). Alternatives to medication and specific questions and reassurances are aspects of quality which have not been previously reported.

CONCLUSIONS

We have demonstrated the feasibility of using participatory research methods in order to develop user-generated quality standards. The development of patient-generated quality standards may offer a more formal method of incorporating the views of service users into quality improvement initiatives. This method can be adapted for generating quality standards applicable to other patient groups.

Selection of hospital antimicrobial prescribing quality indicators: a consensus among German antibiotic stewardship (ABS) networkers

PURPOSE

Simple, valid, and evidence-based indicators to measure the quality of antimicrobial prescribing in acute-care hospitals are urgently needed and increasingly requested by policymakers. The aim of this study was to develop new consensus quality indicators (QIs) for hospital antibiotic stewardship (ABS) and infection management which will be further evaluated for internal quality management and external quality assessment in Germany.

METHODS

Based on an extensive literature review, the Austrian-German hospital ABS Guideline Committee and selected members of the German ABS Expert Network discussed and drafted a list of 99 potential indicators for hospitals that reflect structural prerequisites for ABS (35 items), ABS core activities (18 items), additional ABS measures (5 items), and process of care indicators (both generic and disease-specific-12 and 29 items, respectively). Questionnaires were mailed to German ABS experts and healthcare professionals with further education in ABS. Participants scored (on a nine-point Likert scale) relevance (clinical, ecological/resistance, economical/expenses) and presumed practicability (six categories: clarity of definition, effort to collect data, barrier to implementation, verifiability, suitability for external quality assessment, quality gap), taking into account their local work environment. The scores were processed according to the RAND/UCLA appropriateness method, and QIs were judged relevant if the median (clinical + ecological and/or economical) scores were >6. The indicators thus assessed to be potentially relevant were then filtered according to their practicability. Highly relevant QIs with borderline practicability scores and items with disagreements and overlapping areas were re-discussed in a final multidisciplinary panel consensus workshop convened in November 2012.

RESULTS

Of the 340 questionnaires that were mailed, 75 questionnaires were completed and returned. Of 99 initially proposed items, 32 were excluded due to insufficient scores. Of the remaining 67 items, 21 structural and 21 process of care QIs were finally selected, including four QIs with high clinical and ecological but limited economical relevance, and three QIs with high clinical and economical but limited ecological relevance. Among the selected QIs, efforts to collect data and implementation barriers were scored as suboptimal in many cases.

CONCLUSIONS

A catalog of consensus structural and process of care ABS-QIs was established. These should undergo further pilot and feasibility studies in the German hospital healthcare sector. The panelists were most critical regarding resource use/complexity issues and presumed implementation barriers. How this may limit applicability of QIs remains to be determined.

Clinical stakeholders' opinions on the use of selective decontamination of the digestive tract in critically ill patients in intensive care units: an international Delphi study

Introduction

Selective decontamination of the digestive tract (SDD) is a prophylactic antibiotic regimen that is not widely used in practice. We aimed to describe the opinions of key ‘stakeholders’ about the validity of the existing evidence base, likely consequences of implementation, relative importance of their opinions in influencing overall practice, likely barriers to implementation and perceptions of the requirement for further research to inform the decision about whether to embark on a further large randomised controlled trial.

Methods

This was a Delphi study informed by comprehensive framework of possible determinants of health professionals’ behaviour to study Critical Care practice in four countries. There were four key stakeholder participant groups including ICU physicians, pharmacists, clinical leads, and clinical microbiologists/ infectious disease physicians. Round one comprised participant interviews and Rounds two and three were online questionnaires using Delphi method.

Results

In this study, 141 participants were recruited of whom 82% were retained. Participants rated themselves as knowledgeable about SDD. Antibiotic resistance was identified as the most important issue. SDD was seen as a low clinical priority but few participants reported strong opposition. There was moderate agreement that research to date has not adequately addressed concerns about antibiotic resistance and lacks generalizability. Participants indicated equipoise with regard to benefits and harms of SDD, and indicated strong support for a further randomised trial.

Conclusions

Clinicians have clinical equipoise about the effectiveness of SDD. Future research requires longer follow up to assess antibiotic resistance as well as greater validity/generalizability to provide definitive answers on the effectiveness of decontamination and effects on antibiotic resistance. SDD was regarded as not being a high clinical priority, which may limit future trial participation. These results have identified that further large randomised controlled trial of SDD in critical care is both warranted and appropriate.

Evaluating the quality of colorectal cancer care across the interface of healthcare sectors

Background

Colorectal cancer (CRC) has a high prevalence in western countries. Diagnosis and treatment of CRC is complex and requires multidisciplinary collaboration across the interface of health care sectors. In Germany, a new nationwide established program aims to provide quality information of healthcare delivery across different sectors. Within this context, this study describes the development of a set of quality indicators charting the whole pathway of CRC-care including data specifications that are necessary to operationalize these indicators before practice testing.

Methods

Indicators were developed following a systematic 10 step modified ‘RAND/UCLA Appropriateness Method’ which involved a multidisciplinary panel of thirteen participants. For each indicator in the final set, data specifications relating to sources of quality information, data collection procedures, analysis and feedback were described.

Results

The final indicator set included 52 indicators covering diagnostic procedures (11 indicators), therapeutic management (28 indicators) and follow-up (6 indicators). In addition, 7 indicators represented patient perspectives. Primary surgical tumor resection and pre-operative radiation (rectum carcinoma only) were perceived as most useful tracer procedures initiating quality data collection. To assess the quality of CRC care across sectors, various data sources were identified: medical records, administrative inpatient and outpatient data, sickness-funds billing code systems and patient survey.

Conclusion

In Germany, a set of 52 quality indicators, covering necessary aspects across the interfaces and pathways relevant to CRC-care has been developed. Combining different sectors and sources of health care in quality assessment is an innovative and challenging approach but reflects better the reality of the patient pathway and experience of CRC-care.

Involving patients in quality indicator development - a systematic review

BACKGROUND

Quality indicators (QI) are used in many health care areas to measure, compare, and improve the quality of care. Ideas of quality differ between health care providers and patients, yet patients are not regularly involved in QI development nor does a methodological standard for patient involvement in QI development exist. In this study we systematically reviewed the medical journal articles and gray literature for published approaches for involving patients in QI development.

METHODS

We searched medical literature databases (Medline, Excerpta Medica database, and Cumulative Index to Nursing and Allied Health Literature), screened websites, and contacted experts in the field of QI development for publications on approaches to patient involvement in QI development.

RESULTS

Eleven relevant journal articles and four web-published documents were included. Four major approaches to patient involvement were extracted from the literature: (1) focus group interviews, (2) self-administered questionnaires, (3) individual interviews, and (4) participation in panels during systematic consensus processes. Patients' views were collected by involving patients, patient representatives, or family members.

CONCLUSION

Although there is a large body of literature on QI, publications that describe approaches to patient involvement in QI development are scarce. In principle, indirect and direct methods of patient involvement can be distinguished, and it seems most promising to combine different approaches. However, the limited number of publications identified clearly shows that further research in this field is overdue and that the quality of reporting found in studies within this field needs to be improved.

Quality standards for child and adolescent mental health in primary care

BACKGROUND

Child and adolescent mental health problems are common in primary healthcare settings. However, few parents of children with mental health problems express concerns about these problems during consultations. Based on parental views, we aimed to create quality of care measures for child and adolescent mental health in primary care and develop consensus about the importance of these quality standards within primary care.

METHODS

Quality Standards were developed using an iterative approach involving four phases: 1) 34 parents with concerns about their child's emotional health or behaviour were recruited from a range of community settings including primary care practices to participate in focus group discussions, followed by validation groups or interviews. 2) Preliminary Quality Standards were generated that fully represented the parents' experiences and were refined following feedback from an expert parent nominal group. 3) 55 experts, including parents and representatives from voluntary organisations, across five panels participated in a modified two-stage Delphi study to develop consensus on the importance of the Quality Standards. The panels comprised general practitioners, other community-based professionals, child and adolescent psychiatrists, other child and adolescent mental health professionals and public health and policy specialists. 4) The final set of Quality Standards was piloted with 52 parents in primary care.

RESULTS

In the Delphi process, all five panels agreed that 10 of 31 Quality Standards were important. Although four panels rated 25-27 statements as important, the general practitioner panel rated 12 as important. The final 10 Quality Standards reflected healthcare domains involving access, confidentiality for young people, practitioner knowledge, communication, continuity of care, and referral to other services. Parents in primary care agreed that all 10 statements were important.

CONCLUSIONS

It is feasible to develop a set of Quality Standards to assess mental healthcare provision for children and adolescents seen within primary healthcare services. Primary care practitioners should be aware of parental perspectives about quality of care as these may influence help-seeking behaviours.

Methods for the guideline-based development of quality indicators--a systematic review

BACKGROUND

Quality indicators (QIs) are used in many healthcare settings to measure, compare, and improve quality of care. For the efficient development of high-quality QIs, rigorous, approved, and evidence-based development methods are needed. Clinical practice guidelines are a suitable source to derive QIs from, but no gold standard for guideline-based QI development exists. This review aims to identify, describe, and compare methodological approaches to guideline-based QI development.

METHODS

We systematically searched medical literature databases (Medline, EMBASE, and CINAHL) and grey literature. Two researchers selected publications reporting methodological approaches to guideline-based QI development. In order to describe and compare methodological approaches used in these publications, we extracted detailed information on common steps of guideline-based QI development (topic selection, guideline selection, extraction of recommendations, QI selection, practice test, and implementation) to predesigned extraction tables.

RESULTS

From 8,697 hits in the database search and several grey literature documents, we selected 48 relevant references. The studies were of heterogeneous type and quality. We found no randomized controlled trial or other studies comparing the ability of different methodological approaches to guideline-based development to generate high-quality QIs. The relevant publications featured a wide variety of methodological approaches to guideline-based QI development, especially regarding guideline selection and extraction of recommendations. Only a few studies reported patient involvement.

CONCLUSIONS

Further research is needed to determine which elements of the methodological approaches identified, described, and compared in this review are best suited to constitute a gold standard for guideline-based QI development. For this research, we provide a comprehensive groundwork.

Conducting online expert panels: a feasibility and experimental replicability study

Background

This paper has two goals. First, we explore the feasibility of conducting online expert panels to facilitate consensus finding among a large number of geographically distributed stakeholders. Second, we test the replicability of panel findings across four panels of different size.

Method

We engaged 119 panelists in an iterative process to identify definitional features of Continuous Quality Improvement (CQI). We conducted four parallel online panels of different size through three one-week phases by using the RAND's ExpertLens process. In Phase I, participants rated potentially definitional CQI features. In Phase II, they discussed rating results online, using asynchronous, anonymous discussion boards. In Phase III, panelists re-rated Phase I features and reported on their experiences as participants.

Results

66% of invited experts participated in all three phases. 62% of Phase I participants contributed to Phase II discussions and 87% of them completed Phase III. Panel disagreement, measured by the mean absolute deviation from the median (MAD-M), decreased after group feedback and discussion in 36 out of 43 judgments about CQI features. Agreement between the four panels after Phase III was fair (four-way kappa = 0.36); they agreed on the status of five out of eleven CQI features. Results of the post-completion survey suggest that participants were generally satisfied with the online process. Compared to participants in smaller panels, those in larger panels were more likely to agree that they had debated each others' view points.

Conclusion

It is feasible to conduct online expert panels intended to facilitate consensus finding among geographically distributed participants. The online approach may be practical for engaging large and diverse groups of stakeholders around a range of health services research topics and can help conduct multiple parallel panels to test for the reproducibility of panel conclusions.

Assessment of a novel hybrid Delphi and Nominal Groups technique to evaluate quality indicators

OBJECTIVE

To test the implementation of a novel structured panel process in the evaluation of quality indicators.

DATA SOURCE

National panel of 64 clinicians rating usefulness of indicator applications in 2008-2009.

STUDY DESIGN

Hybrid panel combined Delphi Group and Nominal Group (NG) techniques to evaluate 81 indicator applications.

PRINCIPAL FINDINGS

The Delphi Group and NG rated 56 percent of indicator applications similarly. Group assignment (Delphi versus Nominal) was not significantly associated with mean ratings, but specialty and research interests of panelists, and indicator factors such as denominator level and proposed use were. Rating distributions narrowed significantly in 20.8 percent of applications between review rounds.

CONCLUSIONS

The hybrid panel process facilitated information exchange and tightened rating distributions. Future assessments of this method might include a control panel.

Do the organizational reforms of general practice care meet users' concerns? The contribution of the Delphi method

CONTEXT

The debate over primary care reform in France, as in most OECD countries, centres on questions about efficacy and accessibility. Do these reforms actually respond to the users' concerns?

OBJECTIVE

The objective of this study was to identify the importance that users attribute to different aspects of general practice (GP) care.

DESIGN

The method used was a variant of the classical Delphi approach, called Delphi 'ranking-type'. Between May and September 2009, 74 experts aged over 18 were recruited by 'snowballing' sampling. Three iterative rounds were required to identify the core aspects through a consensus-building approach.

RESULTS

It is shown that users attribute a very high importance to the 'doctor-patient relationship' dimension. The following aspects 'GP patient information about his/her illness', 'Clarity of communication and explanation', and 'Whether the GP seemed listen to the patient' were evaluated by 96% of the experts as being of high importance. The coordination of GP was also considered as a very important aspect for 85% of the experts. In contrast, the aspects that belong to the organizational dimension appeared to be of relatively low importance for users.

CONCLUSIONS

Our results support a comprehensive approach of care and argue in favour of care reorganization following the patient-centred model. To promote organizational care reforms through the prism of the doctor-patient relationship could thus be a fruitful way to insure a better quality of care and the social acceptability of the reforms.

How to optimize tuberculosis case finding: explorations for Indonesia with a health system model

Background

A mathematical model was designed to explore the impact of three strategies for better tuberculosis case finding. Strategies included: (1) reducing the number of tuberculosis patients who do not seek care; (2) reducing diagnostic delay; and (3) engaging non-DOTS providers in the referral of tuberculosis suspects to DOTS services in the Indonesian health system context. The impact of these strategies on tuberculosis mortality and treatment outcome was estimated using a mathematical model of the Indonesian health system.

Methods

The model consists of multiple compartments representing logical movement of a respiratory symptomatic (tuberculosis suspect) through the health system, including patient- and health system delays. Main outputs of the model are tuberculosis death rate and treatment outcome (i.e. full or partial cure). We quantified the model parameters for the Jogjakarta province context, using a two round Delphi survey with five Indonesian tuberculosis experts.

Results

The model validation shows that four critical model outputs (average duration of symptom onset to treatment, detection rate, cure rate, and death rate) were reasonably close to existing available data, erring towards more optimistic outcomes than are actually reported. The model predicted that an intervention to reduce the proportion of tuberculosis patients who never seek care would have the biggest impact on tuberculosis death prevention, while an intervention resulting in more referrals of tuberculosis suspects to DOTS facilities would yield higher cure rates. This finding is similar for situations where the alternative sector is a more important health resource, such as in most other parts of Indonesia.

Conclusion

We used mathematical modeling to explore the impact of Indonesian health system interventions on tuberculosis treatment outcome and deaths. Because detailed data were not available regarding the current Indonesian population, we relied on expert opinion to quantify the parameters. The fact that the model output showed similar results to epidemiological data suggests that the experts had an accurate understanding of this subject, thereby reassuring the quality of our predictions. The model highlighted the potential effectiveness of active case finding of tuberculosis patients with limited access to DOTS facilities in the developing country setting.

Steps, challenges and lessons in developing community mental health care

This paper summarises our own accumulated experience from developing community-orientated mental health services in England and Italy over the last 20-30 years. From this we have provisionally concluded that the following issues are central to the development of balanced mental health services: a) services need to reflect the priorities of service users and carers; b) evidence supports the need for both hospital and community services; c) services need to be provided close to home; d) some services need to be mobile rather than static; e) interventions need to address both symptoms and disabilities; and f) treatment has to be specific to individual needs. In this paper we consider ten key challenges that often face those trying to develop community-based mental health services: a) dealing with anxiety and uncertainty; b) compensating for a possible lack of structure in community services; c) learning how to initiate new developments; d) managing opposition to change within the mental health system; e) responding to opposition from neighbours; f) negotiating financial obstacles; g) avoiding system rigidities; h) bridging boundaries and barriers; i) maintaining staff morale; and j) creating locally relevant ser- vices rather than seeking "the right answer" from elsewhere.