1
|
Cernvall M, Bengtsson J, Bodén R. The Swedish version of the Motivation and Pleasure Scale self-report (MAP-SR): psychometric properties in patients with schizophrenia or depression. Nord J Psychiatry 2024; 78:339-346. [PMID: 38436927 DOI: 10.1080/08039488.2024.2324060] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/22/2021] [Accepted: 02/23/2024] [Indexed: 03/05/2024]
Abstract
PURPOSE Negative symptoms are commonly regarded as a symptom dimension belonging to schizophrenia spectrum disorders but are also present in depression. The recently developed Clinical Assessment Interview for Negative Symptoms (CAINS) has shown to be reliable and valid. A corresponding self-report questionnaire has also been developed, named the Motivation and Pleasure Scale - Self Report (MAP-SR). The purpose was to evaluate the psychometric properties of the Swedish version of the MAP-SR in patients with either schizophrenia or depression. MATERIALS AND METHODS The MAP-SR was translated to Swedish. Participants were 33 patients with schizophrenia spectrum disorders and 52 patients with a depressive disorder and they completed the MAP-SR, the CAINS and other measures assessing adjacent psychopathology, functioning and cognition. RESULTS The internal consistency for the MAP-SR was adequate in both groups (schizophrenia spectrum α = .93, depressive disorder α = .82). Furthermore, the MAP-SR had a large correlation to the motivation and pleasure subscale of the CAINS in patients with schizophrenia disorders (r = -0.75, p < .001), however among patients with depression this correlation was medium-to-large (r = -0.48, p < 0.001). CONCLUSIONS Findings suggest that the Swedish version of the MAP-SR shows promise as a useful measure of motivation and pleasure, especially in patients with schizophrenia spectrum disorders. Furthermore, results also suggest that the MAP-SR does not assess negative symptoms specifically, but that there is an overlap between depressive and negative symptoms.
Collapse
Affiliation(s)
- Martin Cernvall
- Department of Medical Sciences, Psychiatry, Uppsala University, Uppsala, Sweden
| | - Johan Bengtsson
- Department of Medical Sciences, Psychiatry, Uppsala University, Uppsala, Sweden
| | - Robert Bodén
- Department of Medical Sciences, Psychiatry, Uppsala University, Uppsala, Sweden
| |
Collapse
|
2
|
Wehr S, Weigel L, Davis J, Galderisi S, Mucci A, Leucht S. Clinical Assessment Interview for Negative Symptoms (CAINS): A Systematic Review of Measurement Properties. Schizophr Bull 2023:sbad137. [PMID: 37951838 DOI: 10.1093/schbul/sbad137] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/14/2023]
Abstract
BACKGROUND AND HYPOTHESIS Negative symptoms are very important for the overall loss of functioning observed in patients with schizophrenia. There is a need for valid tools to assess these symptoms. STUDY DESIGN We used the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) systematic review guideline to evaluate the quality of the clinical assessment interview for negative symptoms (CAINS) as a clinician-rated outcome measurement (ClinROM). STUDY RESULTS The search strategy resulted in the retrieval of 13 articles, 11 of which were included in this evaluation. In terms of risk of bias, most articles reported on measures of internal consistency and construct validity, which were overall of good quality. Structural validity, reliability, measurement error, and cross-cultural validity were reported with less than optimum quality. There was a risk of bias in ClinROM development. According to the updated criteria of good measurement properties, structural validity, internal consistency, and reliability showed good results. In contrast, hypothesis testing was somewhat poorer. Results for cross-cultural validity were indeterminate. According to the updated GRADE approach from the COSMIN group the scale received a moderate grade. CONCLUSIONS The COSMIN standard allows a judgment of the CAINS as an instrument with the potential to be recommended for use, but which requires further research to assess its quality, in particular in the domains of content validity, internal consistency, and cross-cultural validity.
Collapse
Affiliation(s)
- Sophia Wehr
- Department of Psychiatry and Psychotherapy, School of Medicine, Technical University of Munich, Munich, Germany
| | - Lucia Weigel
- Department of Psychiatry and Psychotherapy, School of Medicine, Technical University of Munich, Munich, Germany
| | - John Davis
- Psychiatric Institute, University of Illinois at Chicago, Chicago, IL
- Maryland Psychiatric Research Center, Baltimore, MD
| | - Silvana Galderisi
- Department of Mental and Physical Health and Preventive Medicine, University of Campania Luigi Vanvitelli, Naples, Italy
| | - Armida Mucci
- Department of Mental and Physical Health and Preventive Medicine, University of Campania Luigi Vanvitelli, Naples, Italy
| | - Stefan Leucht
- Department of Psychiatry and Psychotherapy, School of Medicine, Technical University of Munich, Munich, Germany
| |
Collapse
|
3
|
Bengtsson J, Rad P, Cernvall M, Bodén R. Psychometric properties of the Clinical Assessment Interview for Negative Symptoms (CAINS) in patients with depression and its relationship to affective symptoms. Ann Gen Psychiatry 2023; 22:42. [PMID: 37885022 PMCID: PMC10604520 DOI: 10.1186/s12991-023-00474-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Accepted: 10/20/2023] [Indexed: 10/28/2023] Open
Abstract
BACKGROUND There is a conceptual overlap between negative and depressive symptoms, requiring further exploration to advance the understanding of negative symptoms. The aim of this study was to examine psychometric properties of the Clinical Assessment Interview for Negative Symptoms (CAINS) in patients with depression, and to explore the relationship between the negative and affective symptoms domains. METHODS Fifty-one patients with a depressive episode were included and interviewed with the CAINS and the Brief Psychiatric Rating Scale-Expanded (BPRS-E). Self-reported depressive symptoms were collected with the Montgomery-Asberg Depression Rating Scale (MADRS-S). Inter-rater agreement, internal consistency and validity measures were examined, as were correlations between negative and affective symptoms. RESULTS The intraclass correlation for the CAINS motivation and pleasure subscale (CAINS-MAP) was 0.98 (95% CI 0.96-0.99) and that for the expressional subscale (CAINS-EXP) was 0.81 (95% CI 0.67-0.89). Cronbach's alpha was 0.71 (95% CI 0.57-0.82) for the CAINS-MAP and 0.86 (95% CI 0.79-0.92) for the CAINS-EXP. The correlation with the negative symptoms subscale of the BPRS-E was 0.35 (p = 0.011, blinded/different raters) or 0.55 (p < 0.001, not blinded/same rater). The CAINS-MAP correlated with the affective symptoms subscale of the BPRS-E (r = 0.39, p = 0.005) and the MADRS-S total score (r = 0.50, p < 0.001), but not with anxiety symptoms. CONCLUSIONS Negative symptoms in depression can be assessed with the CAINS with good inter-rater agreement and acceptable internal consistency and validity. There are associations between negative and depressive symptoms that call for further exploration.
Collapse
Affiliation(s)
- Johan Bengtsson
- Department of Medical Sciences, Psychiatry, Uppsala University, Entrance 10, Uppsala University Hospital, 751 85, Uppsala, Sweden.
| | - Parya Rad
- Uppsala University Hospital, Uppsala, Sweden
| | - Martin Cernvall
- Department of Medical Sciences, Psychiatry, Uppsala University, Entrance 10, Uppsala University Hospital, 751 85, Uppsala, Sweden
| | - Robert Bodén
- Department of Medical Sciences, Psychiatry, Uppsala University, Entrance 10, Uppsala University Hospital, 751 85, Uppsala, Sweden
| |
Collapse
|
4
|
Wang LL, Tam MHW, Ho KKY, Hung KSY, Wong JOY, Lui SSY, Chan RCK. Bridge centrality network structure of negative symptoms in people with schizophrenia. Eur Arch Psychiatry Clin Neurosci 2023; 273:589-600. [PMID: 35972557 DOI: 10.1007/s00406-022-01474-w] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/04/2022] [Accepted: 08/02/2022] [Indexed: 11/03/2022]
Abstract
Negative symptoms are complex psychopathology. Although evidence generally supported the NIMH five consensus domains, research seldom examined measurement invariance of this model, and domain-specific correspondence across multiple scales. This study aimed to examine the interrelationship between negative symptom domains captured by different rating scales, and to examine the domain-specific correspondence across multiple scales. We administered the Brief Negative Symptom Scale (BNSS), the Self-evaluation of Negative Symptoms (SNS), and the Scale for Assessment of Negative Symptoms (SANS) to 204 individuals with schizophrenia. We used network analysis to examine the interrelationship between negative symptom domains. Besides regularized partial correlation network, we estimated bridge centrality indices to investigate domain-specific correspondence, while taking each scale as an independent community. The regularized partial correlation network showed that the SNS nodes clustered together, whereas the SANS and the BNSS nodes intermingled together. The SANS attention domain lied at the periphery of the network according to the Fruchterman-Reingold algorithm. The SANS anhedonia-asociality (strength = 1.48; EI = 1.48) and the SANS affective flattening (strength = 1.06; EI = 1.06) had the highest node strength and EI. Moreover, the five nodes of the BNSS bridged the nodes of the SANS and the SNS. BNSS blunted affect (strength = 0.76; EI = 0.76) and SANS anhedonia-asociality (strength = 0.76; EI = 0.74) showed the highest bridge strength and bridge EI. The BNSS captures negative symptoms and bridges the symptom domains measured by the SANS and the SNS. The three scales showed domain-specific correspondence.
Collapse
Affiliation(s)
- Ling-Ling Wang
- Neuropsychology and Applied Cognitive Neuroscience Laboratory, CAS Key Laboratory of Mental Health, Institute of Psychology, Chinese Academy of Sciences, 16 Lincui Road, Beijing, 100101, China
- Department of Psychology, University of Chinese Academy of Sciences, Beijing, China
| | - Michelle H W Tam
- Castle Peak Hospital, Hong Kong Special Administrative Region, China
| | - Karen K Y Ho
- Castle Peak Hospital, Hong Kong Special Administrative Region, China
| | - Karen S Y Hung
- Castle Peak Hospital, Hong Kong Special Administrative Region, China
| | - Jessica O Y Wong
- Castle Peak Hospital, Hong Kong Special Administrative Region, China
| | - Simon S Y Lui
- Department of Psychiatry, School of Clinical Medicine, The University of Hong Kong, Hong Kong Special Administrative Region, China.
| | - Raymond C K Chan
- Neuropsychology and Applied Cognitive Neuroscience Laboratory, CAS Key Laboratory of Mental Health, Institute of Psychology, Chinese Academy of Sciences, 16 Lincui Road, Beijing, 100101, China.
- Department of Psychology, University of Chinese Academy of Sciences, Beijing, China.
| |
Collapse
|
5
|
Giordano GM, Caporusso E, Pezzella P, Galderisi S. Updated perspectives on the clinical significance of negative symptoms in patients with schizophrenia. Expert Rev Neurother 2022; 22:541-555. [PMID: 35758871 DOI: 10.1080/14737175.2022.2092402] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
INTRODUCTION Negative symptoms in schizophrenia are associated with poor response to available treatments, poor quality of life, and functional outcome. Therefore, they represent a substantial burden for people with schizophrenia, their families, and health-care systems. AREAS COVERED In this manuscript, we will provide an update on the conceptualization, assessment, and treatment of this complex psychopathological dimension of schizophrenia. EXPERT OPINION Despite the progress in the conceptualization of negative symptoms and in the development of state-of-the-art assessment instruments made in the last decades, these symptoms are still poorly recognized, and not always assessed in line with current conceptualization. Every effort should be made to disseminate the current knowledge on negative symptoms, on their assessment instruments and available treatments whose efficacy is supported by research evidence. Longitudinal studies should be promoted to evaluate the natural course of negative symptoms, improve our ability to identify the different sources of secondary negative symptoms, provide effective interventions, and target primary and persistent negative symptoms with innovative treatment strategies. Further research is needed to identify pathophysiological mechanisms of primary negative symptoms and foster the development of new treatments.
Collapse
|
6
|
Strauss GP, Raugh IM, Zhang L, Luther L, Chapman HC, Allen DN, Kirkpatrick B, Cohen AS. Validation of accelerometry as a digital phenotyping measure of negative symptoms in schizophrenia. SCHIZOPHRENIA (HEIDELBERG, GERMANY) 2022; 8:37. [PMID: 35853890 PMCID: PMC9261099 DOI: 10.1038/s41537-022-00241-z] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/06/2021] [Accepted: 02/24/2022] [Indexed: 05/05/2023]
Abstract
Negative symptoms are commonly assessed via clinical rating scales; however, these measures have several inherent limitations that impact validity and utility for their use in clinical trials. Objective digital phenotyping measures that overcome some of these limitations are now available. The current study evaluated the validity of accelerometry (ACL), a passive digital phenotyping method that involves collecting data on the presence, vigor, and variability of movement. Outpatients with schizophrenia (SZ: n = 50) and demographically matched healthy controls (CN: n = 70) had ACL continuously recorded from a smartphone and smartband for 6 days. Active digital phenotyping assessments, including surveys related to activity context, were also collected via 8 daily surveys throughout the 6 day period. SZ participants had lower scores on phone ACL variables reflecting vigor and variability of movement compared to CN. ACL variables demonstrated convergent validity as indicated by significant correlations with active digital phenotyping self-reports of time spent in goal-directed activities and clinical ratings of negative symptoms. The discriminant validity of ACL was demonstrated by low correlations with clinical rating scale measures of positive, disorganized, and total symptoms. Collectively, findings suggest that ACL is a valid objective measure of negative symptoms that may complement traditional approaches to assessing the construct using clinical rating scales.
Collapse
Affiliation(s)
| | - Ian M Raugh
- Department of Psychology, University of Georgia, Athens, GA, USA
| | - Luyu Zhang
- Department of Psychology, University of Georgia, Athens, GA, USA
| | - Lauren Luther
- Department of Psychology, University of Georgia, Athens, GA, USA
| | - Hannah C Chapman
- Department of Psychology, University of Georgia, Athens, GA, USA
| | - Daniel N Allen
- Department of Psychology, University of Nevada, Las Vegas, NV, USA
| | - Brian Kirkpatrick
- Department of Psychiatry and Behavioral Sciences, University of Nevada, Reno School of Medicine, Las Vegas, NV, USA
| | - Alex S Cohen
- Department of Psychology, Louisiana State University, Baton Rouge, LA, USA
| |
Collapse
|
7
|
Laraki Y, Lebrun C, Merenciano M, Eisenblaetter M, Attal J, Macgregor A, Decombe A, Capdevielle D, Raffard S. Validation of the French Clinical Assessment Interview for Negative Symptoms in a Sample of Stable French Individuals With Schizophrenia. Front Psychiatry 2022; 13:836600. [PMID: 35432043 PMCID: PMC9010618 DOI: 10.3389/fpsyt.2022.836600] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Accepted: 02/21/2022] [Indexed: 11/13/2022] Open
Abstract
OBJECTIVES The Clinical Assessment Interview for Negative Symptoms (CAINS) is an interview-based instrument evaluating the existence and severity of negative symptoms in people diagnosed with schizophrenia or schizoaffective disorder. The aim of this study is to translate and validate a French version of the CAINS in a French sample of outpatients diagnosed with schizophrenia or schizoaffective disorder. METHODS In this study, we included 84 outpatients with a diagnosis of schizophrenia from the University Department of Adult Psychiatry in Montpellier, France. All participants were assessed for the severity of negative symptoms as well as level of depression. Psychometric properties of the French CAINS were investigated including its factor structure, internal consistency, and interrater and test-retest reliabilities. We also determined the discriminant and convergent validity. RESULTS Exploratory factor analysis and parallel analysis reproduced the two-factor model, and explained 43.55% of the total score variation with good internal consistency (Cronbach α of 0.87). Both interrater and test-retest reliabilities were high for the CAINS and its subscales (intraclass correlation coefficient range, 0.89-0.99). The standard errors of measurement and minimal detectable change were also investigated. Convergent validity of the CAINS was underpinned by correlations obtained with various measures of negative symptoms. Adequate discriminant validity was established by showing that the CAINS did not correlate with positive symptoms. CONCLUSION Overall, our results obtained were similar to those found in the original study of the CAINS. Structural analyses also replicated the two-factor model of the CAINS. Our results indicate that the French CAINS has robust psychometric properties and is a valid tool for evaluating negative symptoms in French-speaking individuals diagnosed with schizophrenia.
Collapse
Affiliation(s)
- Yasmine Laraki
- University Department of Adult Psychiatry, CHU Montpellier, University of Montpellier, Hôpital la Colombière, Montpellier, France.,Université Paul Valéry Montpellier 3, EPSYLON EA, Montpellier, France
| | - Cindy Lebrun
- Université Paul Valéry Montpellier 3, EPSYLON EA, Montpellier, France
| | - Marine Merenciano
- University Department of Adult Psychiatry, CHU Montpellier, University of Montpellier, Hôpital la Colombière, Montpellier, France
| | | | - Jerôme Attal
- University Department of Adult Psychiatry, CHU Montpellier, University of Montpellier, Hôpital la Colombière, Montpellier, France
| | - Alexandra Macgregor
- University Department of Adult Psychiatry, CHU Montpellier, University of Montpellier, Hôpital la Colombière, Montpellier, France
| | - Amandine Decombe
- University Department of Adult Psychiatry, CHU Montpellier, University of Montpellier, Hôpital la Colombière, Montpellier, France
| | - Delphine Capdevielle
- University Department of Adult Psychiatry, CHU Montpellier, University of Montpellier, Hôpital la Colombière, Montpellier, France.,IGF, University of Montpellier, CNRS, INSERM, Montpellier, France
| | - Stéphane Raffard
- University Department of Adult Psychiatry, CHU Montpellier, University of Montpellier, Hôpital la Colombière, Montpellier, France.,Université Paul Valéry Montpellier 3, EPSYLON EA, Montpellier, France
| |
Collapse
|
8
|
Bengtsson J, Bodén R, Neider D, Cernvall M. A blinded validation of the Swedish version of the Clinical Assessment Interview for Negative Symptoms (CAINS). Nord J Psychiatry 2022; 76:44-51. [PMID: 34126848 DOI: 10.1080/08039488.2021.1933174] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Abstract
PURPOSE The Clinical Assessment Interview for Negative Symptoms (CAINS) was developed in order to advance the assessment of negative symptoms. The aim of this study was to validate the Swedish version of the CAINS. MATERIALS AND METHODS Thirty-four out-patients with a schizophrenia spectrum disorder were recruited. All patients were videotaped while interviewed with the CAINS and the Brief Psychiatric Rating Scale (BPRS). Another rater watched the video recordings in the reverse order, enabling a blinded design. The patients also filled in self-reported measures of depression, quality of life, and social and vocational functioning. We calculated inter-rater agreement and internal consistency for the CAINS. We also calculated validity measures by correlating the subscales Motivation and Pleasure (CAINS-MAP) and Expression (CAINS-EXP) to subscales of the BPRS. RESULTS The blinded inter-rater agreement for the CAINS total score was high (ICC = 0.92) but slightly lower for the expression subscale (ICC = 0.76). Cronbach's alpha was 0.84 for the total score. Convergent validity with the negative symptoms subscale of BPRS was different for the blinded and the unblinded data, with a CAINS-MAP correlation of 0.10 (p = 0.580) and a CAINS-EXP correlation of 0.48 (p = 0.004) in the blinded data. The unblinded data had a CAINS-MAP correlation of 0.38 (p = 0.026) and a CAINS-EXP correlation of 0.87 (p < 0.001). Self-rated measures of anhedonia correlated to CAINS-MAP with a coefficient of 0.68 (p < 0.001), while the CAINS-EXP only had a correlation of 0.16 (p = 0.366) to these measures. CONCLUSION The Swedish version of the CAINS displays adequate psychometric properties in line with earlier validation studies.
Collapse
Affiliation(s)
- Johan Bengtsson
- Department of Neuroscience, Psychiatry, Uppsala University, Uppsala, Sweden
| | - Robert Bodén
- Department of Neuroscience, Psychiatry, Uppsala University, Uppsala, Sweden
| | | | - Martin Cernvall
- Department of Neuroscience, Psychiatry, Uppsala University, Uppsala, Sweden
| |
Collapse
|
9
|
Kulenović AD, Mešević ES, Ribić E, Repišti S, Radojičić T, Jovanović N. Factor structure and internal consistency of the Clinical Assessment Interview for Negative Symptoms (CAINS) on a sample of persons with psychotic disorders in Bosnia and Herzegovina. J Neural Transm (Vienna) 2021; 129:905-911. [PMID: 34878600 PMCID: PMC9217762 DOI: 10.1007/s00702-021-02435-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2021] [Accepted: 10/19/2021] [Indexed: 11/17/2022]
Abstract
The assessment of negative symptoms is crucial for development of adequate therapeutic interventions. This is a challenging task due to complex clinical presentation and lack of reliable and valid instruments. This study examined the psychometric characteristics of the Clinical Assessment Interview for Negative Symptoms (CAINS). The sample consisted of 81 persons with schizophrenia or schizoaffective disorder recruited from two health institutions in the Sarajevo Canton: the Clinical Center of the University of Sarajevo and the Psychiatric Hospital of the Sarajevo Canton. The 13 CAINS items grouped into four factors (expression, motivation and satisfaction in the recreational domain, motivation and satisfaction with social relationships, motivation and satisfaction with job and education). The four-factor solution accounted for 87.83% of the variance of manifest items. The reliabilities of extracted factors were as follows: for motivation and satisfaction with social relationships α = 0.897, for motivation and satisfaction with job and education α = 0.961, for Motivation and satisfaction in the recreation domain α = 0.981, and for expression α = 0.938. The highest correlation between factors was found between Motivation and satisfaction with recreation and Motivation and satisfaction with social relationships. On the other hand, the lowest correlation was found between motivation and satisfaction with social relations and motivation and satisfaction with job and education. In conclusion, the study showed that the latent structure of CAINS is adequate, clearly interpretable, and consisted of four factors. The measure can be used for assessment of the negative symptoms in outpatients with psychosis in Bosnia and Herzegovina.
Collapse
Affiliation(s)
- Alma Džubur Kulenović
- Department for Psychiatry, Clinical Center of University of Sarajevo, Sarajevo, Bosnia and Herzegovina.
| | - Eldina Smajic Mešević
- Department for Research and Development, Clinical Center University of Sarajevo, Sarajevo, Bosnia and Herzegovina
| | - Emina Ribić
- Department for Research and Development, Clinical Center University of Sarajevo, Sarajevo, Bosnia and Herzegovina
| | - Selman Repišti
- Clinic for Psychiatry, Clinical Center of Montenegro, Podgorica, Montenegro
| | - Tamara Radojičić
- Clinic for Psychiatry, Clinical Center of Montenegro, Podgorica, Montenegro
| | - Nikolina Jovanović
- Unit for Social and Community Psychiatry, WHO Collaborating Centre for Mental Health Services Development, Bart's and London School of Medicine and Dentistry, Queen Mary University of London, London, UK
| |
Collapse
|
10
|
Galderisi S, Mucci A, Dollfus S, Nordentoft M, Falkai P, Kaiser S, Giordano GM, Vandevelde A, Nielsen MØ, Glenthøj LB, Sabé M, Pezzella P, Bitter I, Gaebel W. EPA guidance on assessment of negative symptoms in schizophrenia. Eur Psychiatry 2021; 64:e23. [PMID: 33597064 PMCID: PMC8080207 DOI: 10.1192/j.eurpsy.2021.11] [Citation(s) in RCA: 79] [Impact Index Per Article: 26.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/06/2023] Open
Abstract
Background During the last decades, a renewed interest for negative symptoms (NS) was brought about by the increased awareness that they interfere severely with real-life functioning, particularly when they are primary and persistent. Methods In this guidance paper, we provide a systematic review of the evidence and elaborate several recommendations for the conceptualization and assessment of NS in clinical trials and practice. Results Expert consensus and systematic reviews have provided guidance for the optimal assessment of primary and persistent negative symptoms; second-generation rating scales, which provide a better assessment of the experiential domains, are available; however, NS are still poorly assessed both in research and clinical settings. This European Psychiatric Association (EPA) guidance recommends the use of persistent negative symptoms (PNS) construct in the context of clinical trials and highlights the need for further efforts to make the definition of PNS consistent across studies in order to exclude as much as possible secondary negative symptoms. We also encourage clinicians to use second-generation scales, at least to complement first-generation ones. The EPA guidance further recommends the evidence-based exclusion of several items included in first-generation scales from any NS summary or factor score to improve NS measurement in research and clinical settings. Self-rated instruments are suggested to further complement observer-rated scales in NS assessment. Several recommendations are provided for the identification of secondary negative symptoms in clinical settings. Conclusions The dissemination of this guidance paper may promote the development of national guidelines on negative symptom assessment and ultimately improve the care of people with schizophrenia.
Collapse
Affiliation(s)
- S Galderisi
- Department of Psychiatry, Campania University Luigi Vanvitelli, Naples, Italy
| | - A Mucci
- Department of Psychiatry, Campania University Luigi Vanvitelli, Naples, Italy
| | - S Dollfus
- CHU de Caen, Service de Psychiatrie, 14000Caen, France.,Normandie Univ, UNICAEN, ISTS EA 7466, GIP Cyceron, 14000Caen, France.,Normandie Univ, UNICAEN, UFR de Médecine, 14000Caen, France
| | - M Nordentoft
- Copenhagen Research Centre for Mental Health (CORE), Copenhagen University Hospital, Copenhagen, Denmark.,Department of Clinical Medicine, Faculty of Health and Medical Science, University of Copenhagen, Copenhagen, Denmark.,Centre for Clinical Intervention and Neuropsychiatric Schizophrenia Research, CINS, Glostrup, Denmark
| | - P Falkai
- Department of Psychiatry, University of Munich, Munich, Germany
| | - S Kaiser
- Division of Adult Psychiatry, Department of Psychiatry, Geneva University Hospitals, Geneva, Switzerland
| | - G M Giordano
- Department of Psychiatry, Campania University Luigi Vanvitelli, Naples, Italy
| | - A Vandevelde
- CHU de Caen, Service de Psychiatrie, 14000Caen, France.,Normandie Univ, UNICAEN, ISTS EA 7466, GIP Cyceron, 14000Caen, France.,Normandie Univ, UNICAEN, UFR de Médecine, 14000Caen, France
| | - M Ø Nielsen
- Department of Clinical Medicine, Faculty of Health and Medical Science, University of Copenhagen, Copenhagen, Denmark.,Centre for Clinical Intervention and Neuropsychiatric Schizophrenia Research, CINS, Glostrup, Denmark.,Center for Neuropsychiatric Schizophrenia Research, CNSR, Glostrup, Denmark
| | - L B Glenthøj
- Copenhagen Research Centre for Mental Health (CORE), Copenhagen University Hospital, Copenhagen, Denmark.,Centre for Clinical Intervention and Neuropsychiatric Schizophrenia Research, CINS, Glostrup, Denmark
| | - M Sabé
- Division of Adult Psychiatry, Department of Psychiatry, Geneva University Hospitals, Geneva, Switzerland
| | - P Pezzella
- Department of Psychiatry, Campania University Luigi Vanvitelli, Naples, Italy
| | - I Bitter
- Department of Psychiatry and Psychotherapy, Semmelweis University, Budapest, Hungary
| | - W Gaebel
- Department of Psychiatry and Psychotherapy, Medical Faculty, Heinrich-Heine University, Düsseldorf, Germany
| |
Collapse
|
11
|
Haguiara B, Koga G, Diniz E, Fonseca L, Higuchi CH, Kagan S, Lacerda A, Correll CU, Gadelha A. What is the Best Latent Structure of Negative Symptoms in Schizophrenia? A Systematic Review. SCHIZOPHRENIA BULLETIN OPEN 2021; 2:sgab013. [PMID: 34901862 PMCID: PMC8650068 DOI: 10.1093/schizbullopen/sgab013] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Background Determining the best latent structure of negative symptoms in schizophrenia could benefit assessment tools, neurobiological research, and targeted interventions. However, no review systematically evaluated studies that assessed and validated latent models of negative symptoms. Objective To identify and evaluate existing latent structure models in the literature of negative symptoms and to determine the best model. Method Systematic search of MEDLINE, EMBASE, and Scopus on July 19, 2020, for confirmatory factor analysis models of negative symptoms in patients with schizophrenia. The available evidence was assessed through 2 sets of criteria: (1) study design quality—based on negative symptoms assessment and modeling strategy and (2) psychometric quality and model fit—based on fit indices and factor definition quality. Results In total, 22 studies (n = 17 086) from 9 countries were included. Studies differed greatly regarding symptom scales, setting, and sample size (range = 86–6889). Dimensional models included 2–6 factors (median = 4). Twelve studies evaluated competing models and adopted appropriate instruments to assess the latent structure of negative symptoms. The 5-factor and hierarchical models outperformed unitary, 2-factor, and 3-factor models on all direct comparisons, and most of the analyses derived from the Brief Negative Symptom Scale. Considering the quality criteria proposed, 5-factor and hierarchical models achieved excellent fit in just one study. Conclusions Our review points out that the 5-factor and hierarchical models represent the best latent structure of negative symptoms, but the immaturity of the relevant current literature may affect the robustness of this conclusion. Future studies should address current limitations regarding psychometric properties and also address biological and clinical validity to refine available models.
Collapse
Affiliation(s)
- Bernardo Haguiara
- Laboratory of Integrative Neurosciences, Department of Psychiatry, Escola Paulista de Medicina/Universidade Federal de São Paulo, SP, Brazil
| | - Gabriela Koga
- Laboratory of Integrative Neurosciences, Department of Psychiatry, Escola Paulista de Medicina/Universidade Federal de São Paulo, SP, Brazil
| | - Elton Diniz
- Laboratory of Integrative Neurosciences, Department of Psychiatry, Escola Paulista de Medicina/Universidade Federal de São Paulo, SP, Brazil
- Schizophrenia Program, Department of Psychiatry, Escola Paulista de Medicina/Universidade Federal de São Paulo, SP, Brazil
| | - Lais Fonseca
- Laboratory of Integrative Neurosciences, Department of Psychiatry, Escola Paulista de Medicina/Universidade Federal de São Paulo, SP, Brazil
- Schizophrenia Program, Department of Psychiatry, Escola Paulista de Medicina/Universidade Federal de São Paulo, SP, Brazil
| | - Cinthia H Higuchi
- Laboratory of Integrative Neurosciences, Department of Psychiatry, Escola Paulista de Medicina/Universidade Federal de São Paulo, SP, Brazil
| | - Simão Kagan
- Laboratory of Integrative Neurosciences, Department of Psychiatry, Escola Paulista de Medicina/Universidade Federal de São Paulo, SP, Brazil
| | - Acioly Lacerda
- Laboratory of Integrative Neurosciences, Department of Psychiatry, Escola Paulista de Medicina/Universidade Federal de São Paulo, SP, Brazil
- Schizophrenia Program, Department of Psychiatry, Escola Paulista de Medicina/Universidade Federal de São Paulo, SP, Brazil
| | - Christoph U Correll
- Department of Psychiatry, The Zucker Hillside Hospital, Northwell Health, Glen Oaks, NY, USA
- Department of Psychiatry and Molecular Medicine, Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, USA
- Department of Child and Adolescent Psychiatry, Charité Universitätsmedizin Berlin, Berlin, Germany
| | - Ary Gadelha
- Laboratory of Integrative Neurosciences, Department of Psychiatry, Escola Paulista de Medicina/Universidade Federal de São Paulo, SP, Brazil
- Schizophrenia Program, Department of Psychiatry, Escola Paulista de Medicina/Universidade Federal de São Paulo, SP, Brazil
| |
Collapse
|
12
|
Ristić I, Jerotić S, Zebić M, Savić B, Vuković V, Russo M, Voskresenski T, Jovanović N, Marić NP. Factorial Structure of the Serbian Version of the Clinical Assessment Interview for Negative Symptoms - Evidence for Three Factors of Negative Symptoms. Front Psychol 2020; 11:570356. [PMID: 33192856 PMCID: PMC7649283 DOI: 10.3389/fpsyg.2020.570356] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2020] [Accepted: 09/28/2020] [Indexed: 11/26/2022] Open
Abstract
Introduction: Negative symptoms are a common occurrence in patients with psychosis spectrum disorders. Previous analysis of the latent structure of the Clinical Assessment Interview for Negative Symptoms (CAINS) - which was developed to advance the assessment of negative symptomatology - showed two underlying sub-domains (Motivation and Pleasure; Expression). Recent findings indicate that a more complex structure might be more applicable. Aim: To evaluate the psychometric properties of the Serbian version of the CAINS in a sample of outpatients (N = 67) with psychosis spectrum disorders. Materials and Methods: Negative symptoms and general level of psychopathology were assessed with Serbian translations of the CAINS, the 53-item version of the Brief Symptom Inventory (BSI), and the 24-item version of the Brief Psychiatric Rating Scale (BPRS). Principal component analysis (PCA) was carried out on the CAINS items, and correlation analyses were done to assess its convergent and discriminant validity. Results: Our results showed an excellent internal consistency (Cronbach's alpha = 0.92). PCA revealed a three-component solution consisting of Expressiveness and Motivation for Social and Family Relationships (Factor 1), Motivation for Vocational Activities (Factor 2), and Motivation for Recreation (Factor 3). Convergent validity was supported by significant correlations with the Negative symptoms domain of the BPRS (Factor 1, 0.695, p < 0.01; Factor 2, 0.352, p < 0.05; Factor 3, 0.452, p < 0.01). When assessing discriminant validity, weak correlations were found with BPRS and BSI scores. Conclusion: The Serbian version of CAINS is a valid, reliable and useful tool for the assessment of negative symptomatology. Our findings support a three-factor structure of CAINS, which indicates that the construct is more complex than envisaged by the original conceptualization of two distinct factors.
Collapse
Affiliation(s)
- Ivan Ristić
- Faculty of Medicine, University of Belgrade, Belgrade, Serbia
- Clinic for Psychiatry, Clinical Centre of Serbia, Belgrade, Serbia
| | - Stefan Jerotić
- Faculty of Medicine, University of Belgrade, Belgrade, Serbia
- Clinic for Psychiatry, Clinical Centre of Serbia, Belgrade, Serbia
| | - Mirjana Zebić
- Faculty of Medicine, University of Belgrade, Belgrade, Serbia
- Clinic for Psychiatry, Clinical Centre of Serbia, Belgrade, Serbia
| | - Bojana Savić
- Clinic for Psychiatry, Clinical Centre of Serbia, Belgrade, Serbia
| | - Vuk Vuković
- Special Hospital for Psychiatric Diseases 'Dr Slavoljub Bakalović', Psychiatry, Vršac, Serbia
| | - Manuela Russo
- Unit for Social and Community Psychiatry - WHO Collaborating Centre for Mental Health Services Development, Queen Mary University of London, London, United Kingdom
| | - Tatjana Voskresenski
- Special Hospital for Psychiatric Diseases 'Dr Slavoljub Bakalović', Psychiatry, Vršac, Serbia
| | - Nikolina Jovanović
- Unit for Social and Community Psychiatry - WHO Collaborating Centre for Mental Health Services Development, Queen Mary University of London, London, United Kingdom
| | - Nađa P Marić
- Faculty of Medicine, University of Belgrade, Belgrade, Serbia
- Institute of Mental Health, Psychiatry, Belgrade, Serbia
| |
Collapse
|
13
|
Strauss GP, Esfahlani FZ, Galderisi S, Mucci A, Rossi A, Bucci P, Rocca P, Maj M, Kirkpatrick B, Ruiz I, Sayama H. Network Analysis Reveals the Latent Structure of Negative Symptoms in Schizophrenia. Schizophr Bull 2019; 45:1033-1041. [PMID: 30256991 PMCID: PMC6737465 DOI: 10.1093/schbul/sby133] [Citation(s) in RCA: 79] [Impact Index Per Article: 15.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Prior studies using exploratory factor analysis provide evidence that negative symptoms are best conceptualized as 2 dimensions reflecting diminished motivation and expression. However, the 2-dimensional model has yet to be evaluated using more complex mathematical techniques capable of testing structure. In the current study, network analysis was applied to evaluate the latent structure of negative symptoms using a community-detection algorithm. Two studies were conducted that included outpatients with schizophrenia (SZ; Study 1: n = 201; Study 2: n = 912) who were rated on the Brief Negative Symptom Scale (BNSS). In both studies, network analysis indicated that the 13 BNSS items divided into 6 negative symptom domains consisting of anhedonia, avolition, asociality, blunted affect, alogia, and lack of normal distress. Separation of these domains was statistically significant with reference to a null model of randomized networks. There has been a recent trend toward conceptualizing the latent structure of negative symptoms in relation to 2 distinct dimensions reflecting diminished expression and motivation. However, the current results obtained using network analysis suggest that the 2-dimensional conceptualization is not complex enough to capture the nature of the negative symptom construct. Similar to recent confirmatory factor analysis studies, network analysis revealed that the latent structure of negative symptom is best conceptualized in relation to the 5 domains identified in the 2005 National Institute of Mental Health consensus development conference (anhedonia, avolition, asociality, blunted affect, and alogia) and potentially a sixth domain consisting of lack of normal distress. Findings have implications for identifying pathophysiological mechanisms and targeted treatments.
Collapse
Affiliation(s)
| | - Farnaz Zamani Esfahlani
- Department of Systems Science and Industrial Engineering & Center for Collective Dynamics of Complex Systems, Binghamton University, Binghamton, NY
| | - Silvana Galderisi
- Department of Psychiatry, University of Campania Luigi Vanvitelli, Naples, Italy
| | - Armida Mucci
- Department of Psychiatry, University of Campania Luigi Vanvitelli, Naples, Italy
| | - Alessandro Rossi
- Section of Psychiatry, Department of Biotechnological and Applied Clinical Sciences, University of L'Aquila, L'Aquila, Italy
| | - Paola Bucci
- Department of Psychiatry, University of Campania Luigi Vanvitelli, Naples, Italy
| | - Paola Rocca
- Section of Psychiatry, Department of Neuroscience, University of Turin, Turin, Italy
| | - Mario Maj
- Department of Psychiatry, University of Campania Luigi Vanvitelli, Naples, Italy
| | - Brian Kirkpatrick
- Department of Psychiatry and Behavioral Sciences, University of Nevada, Reno School of Medicine, Reno, NV
| | - Ivan Ruiz
- Department of Psychology, University of Georgia, Athens, GA
| | - Hiroki Sayama
- Department of Systems Science and Industrial Engineering & Center for Collective Dynamics of Complex Systems, Binghamton University, Binghamton, NY
| |
Collapse
|
14
|
Assessing negative symptoms in schizophrenia: Validity of the clinical assessment interview for negative symptoms in Singapore. Schizophr Res 2019; 206:177-182. [PMID: 30558979 DOI: 10.1016/j.schres.2018.11.029] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/16/2018] [Revised: 07/25/2018] [Accepted: 11/23/2018] [Indexed: 11/20/2022]
Abstract
This study aimed to examine the validity of the Clinical Assessment Interview for Negative Symptoms (CAINS) in Singapore. 274 participants with schizophrenia were assessed on the CAINS, Scale for the Assessment of Negative Symptoms (SANS), Positive and Negative Syndrome Scale (PANSS), Calgary Depression Scale for Schizophrenia (CDSS), Social and Occupational Functioning Assessment Scale (SOFAS) and the Simpson-Angus Extrapyramidal Side Effects Scale (SES). Factor analyses were conducted and Cronbach's coefficient alpha was calculated. Spearman's correlation coefficient was used to assess correlations. The 2-factor model of the CAINS failed to fit our data. Exploratory factor analysis of a randomly selected split-half of the sample yielded four factors: motivation-pleasure (MAP) social, MAP vocational, MAP recreational and expression (EXP), accounting for 73.94% of the total variance. Confirmatory factor analysis on the remaining sample supported this factor structure. Cronbach's alpha for the CAINS was 0.770. Significant correlations were observed between the CAINS total and the SANS total and PANSS negative subscale scores. Good divergent validity was shown by insignificant correlations with PANSS positive subscale score and CDSS total score. The MAP social and recreational factor scores had moderate correlations with the SANS anhedonia-asociality subscale scores, whereas the MAP vocational factor had the highest correlation with the avolition-apathy subscale of the SANS. EXP factor score correlated strongly with the SANS affective flattening and alogia subscales scores. In conclusion, the CAINS has good psychometric properties and can be used by clinicians to assess negative symptoms in individuals with schizophrenia in the local population.
Collapse
|
15
|
Xie DJ, Shi HS, Lui SSY, Shi C, Li Y, Ho KKY, Hung KSY, Li WX, Yi ZH, Cheung EFC, Kring AM, Chan RCK. Cross Cultural Validation and Extension of the Clinical Assessment Interview for Negative Symptoms (CAINS) in the Chinese Context: Evidence from a Spectrum Perspective. Schizophr Bull 2018; 44:S547-S555. [PMID: 29471331 PMCID: PMC6188520 DOI: 10.1093/schbul/sby013] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
The Clinical Assessment Interview for Negative Symptoms (CAINS) was designed in accordance with the recent theory and research in social affective neuroscience and to address the psychometric and conceptual limitations of other instruments assessing negative symptoms. The present study aimed to provide a large-scale validation of the CAINS in China and examine its applicability and validity evidence across the schizophrenia spectrum. Using confirmatory factor analysis, our results replicated the original findings in the US development samples that the CAINS possesses a stable 2-factor structure, namely "motivation/pleasure" and "expression". We also found significant correlations between the CAINS and other negative symptom measures. The CAINS demonstrated good discriminant validity in differentiating negative symptoms in people with schizophrenia, nonpsychotic first-degree relatives and people with social anhedonia. People with schizophrenia exhibited significantly higher CAINS subscale scores than first-degree relatives and healthy controls. In addition, first-degree relatives had higher "motivation/pleasure" scores than healthy controls. The "motivation/pleasure" subscale scores of individuals with social anhedonia were also significantly higher than healthy controls.
Collapse
Affiliation(s)
- Dong-jie Xie
- Neuropsychology and Applied Cognitive Neuroscience Laboratory, CAS Key Laboratory of Mental Health, Institute of Psychology, Beijing, China,Department of Psychology, University of Chinese Academy of Sciences, Beijing, China
| | - Hai-song Shi
- Neuropsychology and Applied Cognitive Neuroscience Laboratory, CAS Key Laboratory of Mental Health, Institute of Psychology, Beijing, China,Department of Psychology, University of Chinese Academy of Sciences, Beijing, China,North China Electric Power University, Beijing, China
| | - Simon S Y Lui
- Neuropsychology and Applied Cognitive Neuroscience Laboratory, CAS Key Laboratory of Mental Health, Institute of Psychology, Beijing, China,Castle Peak Hospital, Hong Kong Special Administrative Region, China
| | - Chuan Shi
- Peking University Sixth Hospital, Beijing, China,Peking University Institute of Mental Health, Beijing, China,Key Laboratory of Mental Health, Ministry of Health (Peking University), Beijing, China
| | - Ying Li
- Neuropsychology and Applied Cognitive Neuroscience Laboratory, CAS Key Laboratory of Mental Health, Institute of Psychology, Beijing, China,Department of Psychology, University of Chinese Academy of Sciences, Beijing, China,Haidian District Mental Health Prevent-Treatment Hospital, Beijing, China
| | - Karen K Y Ho
- Castle Peak Hospital, Hong Kong Special Administrative Region, China
| | - Karen S Y Hung
- Castle Peak Hospital, Hong Kong Special Administrative Region, China
| | - Wen-xiu Li
- Haidian District Mental Health Prevent-Treatment Hospital, Beijing, China
| | - Zheng-hui Yi
- Shanghai Mental Health Centre, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Eric F C Cheung
- Castle Peak Hospital, Hong Kong Special Administrative Region, China
| | - Ann M Kring
- Department of Psychology, University of California, Berkeley, CA
| | - Raymond C K Chan
- Neuropsychology and Applied Cognitive Neuroscience Laboratory, CAS Key Laboratory of Mental Health, Institute of Psychology, Beijing, China,Department of Psychology, University of Chinese Academy of Sciences, Beijing, China,To whom correspondence should be addressed; Institute of Psychology, Chinese Academy of Sciences, 16 Lincui Road, Beijing 100101, China; tel/fax: +86(0)10-64836274, e-mail:
| |
Collapse
|
16
|
Jo HS, Song TM, Kim BG. Analysis of the Factors Affecting Consumer Acceptance of Accredited Online Health Information. J Korean Med Sci 2017; 32:1757-1763. [PMID: 28960026 PMCID: PMC5639054 DOI: 10.3346/jkms.2017.32.11.1757] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/10/2017] [Accepted: 08/12/2017] [Indexed: 11/20/2022] Open
Abstract
With the increasing use of the internet and the spread of smartphones, health information seekers obtain considerable information through the internet. As the amount of online health information increases, the need for quality management of health information has been emphasized. The purpose of this study was to investigate the factors affecting the intention of using accredited online health information by applying the extended technology acceptance model (Extended-TAM). An online survey was conducted from September 15, 2016 to October 3, 2016, on 500 men and women aged 19-69 years. The results showed that the greatest factor influencing the acceptance of the accredited health information was perceived usefulness, and the expectation for the quality of the accreditation system was the most important mediator variable. In order to establish the health information accreditation system as a means to provide easy and useful information to the consumers, it is necessary to carry out quality management and promote the system through the continuous monitoring of the accreditation system.
Collapse
Affiliation(s)
- Heui Sug Jo
- Department of Health Policy and Management, Kangwon National University College of Medicine, Chuncheon, Korea
| | - Tae Min Song
- Department of Health Management, Sahmyook University, Seoul, Korea
| | - Bong Gi Kim
- Korea Institute of Drug Safety and Risk Management, Anyang, Korea.
| |
Collapse
|