Differences between patients in whom physicians agree versus disagree about the preoperative diagnosis of heart failure

STUDY OBJECTIVE

To quantify preoperative heart failure (HF) diagnostic agreement and identify characteristics of patients in whom physicians agreed versus disagreed about the diagnosis.

DESIGN

Observational cohort study.

SETTING

Patients undergoing major non-cardiac surgery at an academic center between 2015 and 2019.

PATIENTS

40,659 patients undergoing major non-cardiac surgery, among which a stratified subsample of 1018 patients with and without documented HF was reviewed.

INTERVENTIONS

Via a panel of physicians frequently managing patients with HF (cardiologists, cardiac anesthesiologists, intensivists), detailed chart reviews were performed (two per patient; median review time 32 min per reviewer per patient) to render adjudicated HF diagnoses.

MEASUREMENTS

Adjudicated diagnostic agreement measures (percent agreement, Krippendorf's alpha) and univariate comparisons (standardized differences) between patients in whom physicians agreed versus disagreed about the preoperative HF diagnosis.

MAIN RESULTS

Among patients with documented HF, physicians agreed about the diagnosis in 80.0% of cases (consensus positive), disagreed in 13.8% (disagreement), and refuted the diagnosis in 6.3% (consensus negative). Conversely, among patients without documented HF, physicians agreed about the diagnosis in 88.0% (consensus negative), disagreed in 8.4% (disagreement), and refuted the diagnosis in 3.6% (consensus positive). The estimated agreement for the 40,659 cases was 91.1% (95% CI 88.3%-93.9%); Krippendorff's alpha was 0.77 (0.75-0.80). Compared to patients in whom physicians agreed about a HF diagnosis, patients in whom physicians disagreed exhibited fewer guideline-defined HF diagnostic criteria.

CONCLUSIONS

Physicians usually agree about HF diagnoses adjudicated via chart review, although disagreement is not uncommon and may be partly explained by heterogeneous clinical presentations. Our findings inform preoperative screening processes by identifying patients whose characteristics contribute to physician disagreement via chart review. Clinical Trial Number / Registry URL: Not applicable.

Patient characteristics, validity of clinical diagnoses and Outcomes Associated with Suicidality in Inpatients with Symptoms of Depression (OASIS-D): design, procedures and outcomes

BACKGROUND

Suicidality, ranging from passive suicidal thoughts to suicide attempt, is common in major depressive disorder (MDD). However, relatively little is known about patient, illness and treatment characteristics in those with co-occurring MDD and suicidality, including the timing of and factors associated with the offset, continuation or reemergence of suicidality. Here, we present the background, rationale, design and hypotheses of the Patient Characteristics, Validity of Clinical Diagnoses and Outcomes Associated with Suicidality in Inpatients with Symptoms of Depression (OASIS-D) study, an investigator-initiated, observational study, funded by Janssen-Cilag GmbH.

METHODS/RESULTS

OASIS-D is an eight-site, six-month, cohort study of patients aged 18-75 hospitalized with MDD. Divided into three sub-studies and patient populations (PPs), OASIS-D will (i) systematically characterize approximately 4500 consecutively hospitalized patients with any form of unipolar depressive episode (PP1), (ii) evaluate the validity of the clinical diagnosis of moderate or severe unipolar depressive episode with the Mini-International Neuropsychiatric Interview (M.I.N.I.) and present suicidality (at least passive suicidal thoughts) present ≥ 48 h after admission with the Sheehan-Suicide Tracking Scale (S-STS), assessing also predictors of the diagnostic concordance/discordance of MDD in around 500 inpatients (PP2), and (iii) characterize and prospectively follow for 6 months 315 inpatients with a research-verified moderate or severe unipolar depressive episode and at least passive suicidal thoughts ≥ 48 h after admission, evaluating treatment and illness/response patterns at baseline, hospital discharge, 3 and 6 months. Exploratory objectives will describe the association between the number of days with suicidality and utilization of outpatient and inpatient care services, and structured assessments of factors influencing the risk of self-injurious behavior without suicidal intent, and of continuous, intermittent or remitted suicidality during the 6-month observation period.

CONCLUSION

Despite their frequency and clinical relevance, relatively little is known about patient and treatment characteristics of individuals with MDD and suicidality, including factors moderating and mediating the outcome of both MDD and suicidality. Results of the OASIS-D study are hoped to improve the understanding of the frequency, correlates and 6-month naturalistic treatment and outcome trajectories of different levels of suicidality in hospitalized adults with MDD and suicidality.

TRIAL REGISTRATION

NCT04404309 [ClinicalTrials.gov].

Informative value of referral letters from general practice for child and adolescent mental healthcare

Although referral letters (RLs) form a nodal point in a patient's care journey, little is known about their informative value in child and adolescent mental healthcare. To determine the informative value of RLs to child and adolescent psychiatry, we conducted a chart review in medical records of minors registered at specialized mental healthcare between January 2015 and December 2017 (The Netherlands). Symptoms indicated in RLs originating from general practice (N = 723) were coded and cross-tabulated with the best estimate clinical classifications made in psychiatry. Results revealed that over half of the minors in the sample were classified in concordance with at least one reason for referral. We found fair to excellent discriminative ability for indications made in RLs concerning the most common psychiatric classifications (95% CI AUC: 60.9-70.6 for anxiety disorders to 90.5-100.0 for eating disorders). Logistic regression analyses suggested no statistically significant effects of gender, age, severity or mental healthcare history, with the exception of age and attention deficit hyperactivity disorders (ADHD), as RLs better predicted ADHD with increasing age (OR = 1.14, 95% CI 1.03-1.27). Contextual problems, such as difficulties studying, problems with parents or being bullied were indicated frequently and associated with classifications in various disorder groups. To conclude, general practitioners' RLs showed informative value, contrary to common beliefs. Replication studies are needed to reliably incorporate RLs into the diagnostic work-up.

Drivers of shared decision making in inpatient psychiatry: An exploratory survey of patients' and multi-disciplinary team members' perspectives

OBJECTIVE

To assess the prevalence and predictors of Shared Decision Making (SDM) in an adult, inpatient psychiatric setting.

METHOD

Multi-disciplinary clinician focus groups and patient pre-testing informed the development of a survey on 4 SDM and 11 factors hypothesized to interfere with SDM. The survey was administered to 89 adult inpatients (80% response rate) and their treatment team psychiatrists, nurses, and social workers (n = 338 ratings, 95% response rate). Group differences and predictors were estimated using t and F-tests.

RESULTS

Patients' mean SDM score (n = 64, standardized Cronbach alpha = 0.858) was 3.35 ± 1.13 (5 = highest agreement), and correlated with overall satisfaction with care (n = 61, r = 0.399, p = 0.001). Patients' disagreement with clinician's diagnosis (44% of patients) correlated with lower SDM ratings by patients (t = 2.55, df = 62, p = 0.013) and by clinicians (t = 2.99, df = 69, p = 0.004). Psychotic diagnoses were not a significant determining factor for SDM. Overall, clinicians rated SDM more favorably than patients (t = -5.43, df = 63, p < 0.001), with nurses and social workers rating SDM higher than physicians (p < 0.001).

CONCLUSIONS

Diagnostic agreement / disagreement is a key predictor of SDM for patients and clinicians, while presence of psychosis is not. SDM was rated higher by clinicians than patients. SDM ratings vary significantly between clinical disciplines.

Beyond kappa: an informational index for diagnostic agreement in dichotomous and multivalue ordered-categorical ratings

Agreement measures are useful tools to both compare different evaluations of the same diagnostic outcomes and validate new rating systems or devices. Cohen's kappa (κ) certainly is the most popular agreement method between two raters, and proved its effectiveness in the last sixty years. In spite of that, this method suffers from some alleged issues, which have been highlighted since the 1970s; moreover, its value is strongly dependent on the prevalence of the disease in the considered sample. This work introduces a new agreement index, the informational agreement (IA), which seems to avoid some of Cohen's kappa's flaws, and separates the contribution of the prevalence from the nucleus of agreement. These goals are achieved by modelling the agreement-in both dichotomous and multivalue ordered-categorical cases-as the information shared between two raters through the virtual diagnostic channel connecting them: the more information exchanged between the raters, the higher their agreement. In order to test its fair behaviour and the effectiveness of the method, IA has been tested on some cases known to be problematic for κ, in the machine learning context and in a clinical scenario to compare ultrasound (US) and automated breast volume scanner (ABVS) in the setting of breast cancer imaging. Graphical Abstract To evaluate the agreement between the two raters [Formula: see text] and [Formula: see text] we create an agreement channel, based on Shannon Information Theory, that directly connects the random variables X and Y, that express the raters outcomes. They are the terminals of the chain X⇔ diagnostic test performed by [Formula: see text] ⇔ patient condition[Formula: see text] ⇔ diagnostic test performed by [Formula: see text] ⇔ Y.

[Diagnostic agreement of primary care and minor surgery in a basic health area]

AIM

Minor Surgery (MS) is an ever-increasing programmed activity in Primary Health Care Centres (PHC). The aim of this study is to establish the clinical and histopathology diagnostic agreement between PHC and MS and evaluating the efficacy of this activity.

METHODS AND MATERIALS

A retrospective, observational, and reliability study was performed. A total of 234 patient specimens were sent to Histopathology between January 2014 and December 2018 in basic health area of San Benito-La Laguna, Santa Cruz de Tenerife. Of these, 203 specimens met criteria, with 31 being excluded due to death or absence of diagnosis. Sociodemographic and diagnostic variables were analysed, and 10 possible diagnoses were grouped into 3 categories according to their nature. Cohen kappa coefficient was used as the main evaluation measure.

RESULTS

The majority of specimens were obtained from women (51.2%), and the mean age was 52.82±17.82 years. The most frequently referred lesion was the epidermoid cyst (20.2-21.67%). A clinical-pathological agreement of 60% was obtained in Minor Surgery, with a specificity of 98.3% and a sensitivity of 61.9%. In Primary Care agreement was 36.1%, with a specificity of 98.4% and a sensitivity of 42.8%. Infectious lesions represented the largest concordance difference obtained, with 27% less in Primary Care compared to Minor Surgery.

CONCLUSIONS

Minor Surgery is an effective support in the initial diagnosis of lesions referred for evaluation at Primary Care. However, it is necessary to implement improvements in diagnostic efficacy of Primary Care.

The agreement between the referrer, practitioner and research diagnosis of autistic spectrum conditions among children attending child and adolescent mental health services

We aimed to explore the levels of agreement about the diagnoses of Autistic Spectrum Conditions between the referrer, CAMHS practitioner and a research diagnosis, as well as the stability of the practitioner's diagnosis over time in a secondary analysis of data from 302 children attending two Child and Adolescent Mental Health Services over two years. Kappa coefficient was used to assess the agreement between the referrer and research diagnosis. Kendall's tau b coefficient was used to assess the agreement between the practitioner and the research diagnosis assigned using the Development and Well-Being Assessment, as well as the agreement between the referrer's indication of presenting problems and the practitioner diagnosis. Diagnostic stability was explored in children with and without a research diagnosis of Autistic Spectrum Condition. There was a moderate level of agreement between the referrer and research diagnosis (Kappa = 0.51) and between practitioner's and research diagnosis (Kendall's tau = 0.60) at baseline, which reduced over the subsequent two years. Agreement between the referrer and practitioner's diagnosis at baseline was fair (Kendall's tau = 0.36).The greatest diagnostic instability occurred among children who practitioners considered to have possible Autistic Spectrum Conditions but who did not meet research diagnostic criteria. Further studies could explore the approaches used by practitioners to reach diagnoses and the impact these may have on diagnostic stability in Autistic Spectrum Conditions. Standardised assessment using a clinically rated diagnostic framework has a potential role as an adjunct to standard clinical care and might be particularly useful where practitioners are uncertain.

Teledermatology in Remote Parts of Chile: Experience in 4 Isolated Rural Areas

BACKGROUND

Teledermatology was introduced in Chile to make up for the lack of dermatologists and improve the primary care system's ability to resolve problems. This strategy was implemented in the province of Palena in 2013, but outcomes were not analyzed and reported until now.

MATERIAL AND METHODS

Descriptive, cross-sectional study with analysis of inter-rater agreement on diagnoses. All the teledermatology consultations made on behalf of patients in Ayacara, Chaitén, Futaleufú, and Palena from 2013 through 2017 were analyzed. Data were extracted from the MINSAL platform. The κ statistic was used to assess diagnostic agreement.

RESULTS

A total of 326 teledermatology consultations were made. The mean (SD) age of the patients was 35.8 (22.4) years. The majority (59.8%) were female. Palena generated the largest volume of cases (40.8%). The mean time until a teledermatology diagnosis was given was 12.6 (22.8) days. This response time decreased to 6.4 (5.2) days after a staff dermatologist from Hospital Puerto Montt joined the program (P=.0005). Diagnosis concordance between the general practitioner and the dermatologist was moderate (κ=0.5). Physical examination by the dermatologist at the hospital of reference (Puerto Montt) was necessary for 20.6% of the patients, and time until a definitive diagnosis for the patient took longer in such cases (25.7 [41.2] days) than for the cases that didn't need a physical evaluation at the Hospital of Puerto Montt (P<.0001).

CONCLUSIONS

Teledermatology provides a necessary diagnostic and therapeutic resource for general practitioners in isolated locations. Diagnostic agreement is moderate between the generalist and the specialist, similar to agreement in international studies but lower than agreement reported in other Chilean studies. The reduction in diagnostic response time after incorporation of a dermatologist on staff at the hospital of Puerto Montt was significant. The significant difference between the time for diagnosis under teledermatology and the delay when physical examination is required at the hospital of Puerto Montt identifies an aspect of care management to improve.

Do neurologists around the world agree when diagnosing epilepsy? - Results of an international EpiNet study

OBJECTIVE

Previous studies have shown moderate agreement between physicians when diagnosing epilepsy, but have included small numbers. The EpiNet study group was established to undertake multicentre clinical trials in epilepsy. Before commencing trials, we wanted to determine levels of agreement between physicians from different countries and different health systems when diagnosing epilepsy, specific seizure types and etiologies.

METHODS

30 Case scenarios describing six children and 24 adults with paroxysmal events (21 epileptic seizures, nine non-epileptic attacks) were presented to physicians with an interest in epilepsy. Physicians were asked how likely was a diagnosis of epilepsy; if seizures were generalised or focal; and the likely etiology. For 23 cases, clinical information was presented in Step 1, and investigations in Step 2.

RESULTS

189 Participants from 36 countries completed the 30 cases. Levels of agreement were determined for 154 participants who provided details regarding their clinical experience. There was substantial agreement for diagnosis of epilepsy (kappa=0.61); agreement was fair to moderate for seizure type(s) (kappa=0.40) and etiology (kappa=0.41). For 23 cases with two steps, agreement increased from step 1 to step 2 for diagnosis of epilepsy (kappa 0.56-0.70), seizure type(s) (kappa 0.38-0.52), and etiology (kappa 0.38-0.47). Agreement was better for 53 epileptologists (diagnosis of epilepsy, kappa=0.66) than 56 neurologists with a special interest in epilepsy (kappa=0.58). Levels of agreement differed slightly between physicians practicing in different parts of the world, between child and adult neurologists, and according to one's experience with epilepsy.

CONCLUSION

Although there is substantial agreement when epileptologists diagnose epilepsy, there is less agreement for diagnoses of seizure types and etiology. Further education of physicians regarding semiology of different seizure types is required. Differences in approach to diagnosis, both between physicians and between countries, could impact negatively on clinical trials of anti-epileptic drugs.

Defining Papillary Carcinoma of the Thyroid: A Short Review and Analysis

OBJECTIVES

To review how changes in the pathologic definitions for papillary tumors of the thyroid during recent decades have affected outcomes for patients with these tumors.

METHODS

Forty-nine previous reports or studies involving collectively 53,606 patients were reviewed, and new analyses were performed on the data to include analyses of agreement, incidence, survival, and diagnostic categories.

RESULTS

The past emphasis on cytologic features to define papillary tumors has not resulted in ideal pairwise agreement between pathologists and has produced incidence and survival data suggesting overdetection and overdiagnosis. Most recently, tissue patterns have been reemphasized.

CONCLUSIONS

With the recent reemphasis on diagnostic tissue patterns (over cytologic criteria), agreements between pathologists for the diagnosis of papillary tumors should improve, and the incidence of papillary carcinoma should decline. Nevertheless, updated survival analyses demonstrate excellent long-term survival for most of those diagnosed with papillary carcinomas.

Diagnostic accuracy and feasibility of a net-based application for diagnosing common psychiatric disorders

A net-based application, which enables users to diagnose and manage common psychiatric disorders independently, was developed. Diagnostic accuracy and reliability of the application were compared with the Mini International Neuropsychiatric Interview among 274 adult outpatients. The screening sub-module of the application had high sensitivity, specificity and negative predictive values for most disorders, but low positive predictive values for several disorders. The criteria-based diagnostic sub-module demonstrated moderate to substantial agreement (kappa>0.50) for all 4 broad categories and 10 of the 18 individual disorders included. Diagnostic agreement was, however, low (kappa<0.4) for most of the disorders comprising the broad category of 'neurotic and stress-related disorders'. Low sensitivity was also observed among these disorders, though specificity, and positive and negative predictive values were high for most disorders. Inter-rater reliability of the application's diagnoses was high. Symptom-severity and functional status scores correlated significantly with those on standard scales. Average time taken was 5min for screening and 20min for detailed diagnostic assessment. A majority of the patients, their relatives and interviewers were satisfied with the assessment. The results suggest that with further refinement the application could be suitable for use as a net-based diagnostic tool for psychiatric disorders.

Tongue diagnosis system for quantitative assessment of tongue coating in patients with functional dyspepsia: a clinical trial

ETHNOPHARMACOLOGICAL RELEVANCE

Tongue diagnosis is a significant procedure to examine the physiological and pathological changes of the human body in oriental medicine. However, the conventional method of tongue diagnosis including direct observation of tongue has limitations because of various external factors and subjective factors. Therefore, the current study investigated the usefulness of the tongue diagnosis system (TDS) as a diagnostic tool for evaluating tongue coating thickness (TCT) by assessing the agreement between the TDS and a gold standard established by assessors using the conventional method.

MATERIALS AND METHODS

The present study was designed as a prospective clinical trial including 60 patients with functional dyspepsia. The TDS was used to capture tongue images twice within a 30-min interval to assess its reproducibility. Tongue coating percentage was measured by the TDS, and TCT was classified as either no coating, thin coating, or thick coating according to the existing diagnostic criteria. After both TDS examinations, the weight of tongue coating was quantitatively measured, and the correlation between the actual quantity of tongue coating and the percentage of the coating measured by the TDS was analyzed using Pearson׳s correlation. After collecting tongue images in all 60 patients, tongue coating was evaluated using a conventional method by 5 well-trained assessors to establish the gold standard for evaluating TCT, which allowed us to assess the diagnostic agreement between the TDS and the gold standard. After 2 weeks, TCT evaluation was repeated by the same assessors using the same images but in a random order.

RESULTS

The agreement between the TDS and the gold standard for evaluating TCT was almost perfect (weighted kappa, 0.840), as was the reproducibility of the TDS (weighted kappa, 0.851). The percentage of tongue coating measured by the TDS was significantly correlated with the weight of tongue coating (r=0.442, p <0.001). The levels of intra-rater reliability ranged from substantial to almost perfect (range of weighted kappa, 0.777-0.923). The inter-rater reliability of 5 assessors was moderate (weighted kappa, 0.563).

CONCLUSIONS

The present study demonstrated that the TDS can be used as a diagnostic tool for the objective and standardized evaluation of TCT in actual clinical practice.

[Diagnostic agreement between two glycosylated a1b hemoglobin methods in Primary Care]

OBJECTIVE

Several methods are available for measuring glycosylated A1c hemoglobin (HbA1c), all rapid methods for point of care use in a clinical or laboratory setting. This study attempts to compare the diagnostic agreement between two methods for detection of HbA1c.

MATERIAL AND METHODS

A descriptive cross-sectional study of diagnostic agreement was carried out in the Los Carmenes Health Centre. Two groups of patients -with and without type 2 diabetes- were consecutively included. A method for point-of-care use in a Primary Care Clinic setting (DCA(TM) Systems Siemens(®)) was compared with a laboratory test (chromatographic analysis). An analysis was made of the mean concentration of HbA1c, the agreement between methods, using the intra-class correlation coefficient (CCLA1) and the Bland-Altman method.

RESULTS

A total of 102 patients were included, 62 diabetic (60.8%) and 40 non-diabetic (39.2%). The overall mean HbA1c was 6.46% (SD=0.88) in the analysis using capillary blood in the clinic with the DCA™ system, and 6.44% (SD=0.86) using the laboratory test (P>.05). The degree of agreement between the two tests was 0.975 (95% CI: 0.963-0.983). The mean of the differences between the results of the two assessed tests was 0.024 (SD=0.27). The percentage of points outside the limits of optimal agreement, as defined in the Bland-Altman graph, was 2.5%.

CONCLUSIONS

Diagnostic agreement between a method for point-of-care use in a Primary Health Care Clinic and a laboratory test was very high. Detection at the point-of-care allows a quick and simple assessment of HbA1c.

Skin conditions in primary care: an analysis of referral demand

INTRODUCTION

Skin conditions are among the main reasons for seeking primary health care. Primary care physicians (PCPs) must diagnose skin conditions and determine their impact, and must therefore incorporate the relevant knowledge and skills into their education. The present study analyzes the reasons for primary care referral to dermatology (referral demand) as well as diagnostic agreement between PCPs and dermatologists informed by pathology where appropriate.

MATERIAL AND METHODS

Data were collected for 755 patients and 882 initial dermatology appointments from February 1, 2012 through April 30, 2012 following primary care referral. Data obtained included age, sex, occupation, reason for referral, primary care diagnosis, and dermatologic diagnosis. Statistical analysis of the data for each diagnosed condition identified frequency, reasons for referral, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and the κ statistic for diagnostic agreement.

RESULTS

The most common diagnoses were seborrheic keratosis, melanocytic nevus, actinic keratosis, and acne. The main reason for referral was diagnostic assessment (52.5%). For skin tumors, sensitivity of primary care diagnosis was 22.4%, specificity 94.7%, PPV 40.7%, and NPV 88.3%, with a κ of 0.211. For the more common diagnoses, primary care sensitivity was generally low and specificity high.

CONCLUSIONS

According to our results, primary care physicians are better qualified to rule out a given skin condition in a patient (high specificity) than to establish an accurate clinical diagnosis (poor sensitivity). This suggests that knowledge and skills training should be organized for primary care physicians to improve management of skin conditions-especially skin cancer, because of its impact. A more responsive system would ensue, with shorter waiting lists and better health care.