Feasibility, usefulness, limitations and perspectives of routine outcome assessment: the South Verona Outcome Project

Longitudinal analysis of statistical and clinically significant psychosocial change following mental health rehabilitation

Purpose: With appropriate mental health rehabilitation, schizophrenia is increasingly associated with reports of recovery and stability. However, there is little empirical evidence evaluating the efficacy of services delivering this care. This study evaluated the effectiveness of rehabilitation for improving psychosocial function in consumers with schizophrenia.Methods: An electronic database of standardized assessment instruments mandated and maintained by the health service was retrospectively reviewed to extract ratings of psychosocial function, daily living skills, and mood state from consecutive admissions to an inpatient rehabilitation service. Outcomes were compared at admission, discharge, and one-year follow-up to identify statistically significant change. Individual reliable and clinically significant change was also assessed by comparison with a normative group of clients functioning independently in the community.Results: From admission to discharge the rehabilitation group made statistically significant gains in psychosocial function and daily living skills. Improvements were reliable and clinically significant in one-quarter to one-third of individual consumers. Approximately half sustained their improvements at follow-up, although this represented only a small fraction of the overall cohort. Consumers not demonstrating gains exhibited psychometric floor effects at admission.Conclusions: Rehabilitation can produce statistically and clinically significant immediate improvement in psychosocial function for a sub-set of consumers with elevated scores at admission. The durability of any gains is less clear, and strategies promoting longer-term maintenance are encouraged. Furthermore, currently mandated outcome measures are confounded by issues of sensitivity and reporting compliance, and exploration of alternative instruments for assessing recovery is recommended.Implications for RehabilitationRoutinely collected standardized outcome measures can be used to investigate the effectiveness of mental health rehabilitationIn addition to statistical significance, the clinical significance of outcomes should be evaluated to identify change that is individually meaningfulCurrently mandated outcomes instruments do not adequately evaluate many individuals' recovery journeyMental health service evaluation and quality improvement processes would likely benefit from adoption of recovery-oriented measures.

Morale of mental health professionals in Community Mental Health Services of a Northern Italian Province

Summary

Aims — To explore morale of psychiatrists and psychiatric nurses working in Community Mental Health Centres (CMHC) in an Italian Province, and identify influential factors. Methods — Thirty psychiatrists and 30 nurses working in CMHCs in Modena completed questionnaires on burnout, team identity and job satisfaction. They also answered open questions about different aspects of their work. Answers were subjected to content analysis. Regression analyses were used to identify factors that predicted morale across groups. Results — Psychiatrists had higher scores on emotional exhaustion and depersonalisation. There were no significant differences between the two groups in job satisfaction and job or role perception. Professionals reported positive relationships with patients as the most enjoyable aspects of their job, whilst team conflicts and high workloads were seen as most difficult to cope with. Multivariate analyses showed that being a psychiatrist and perceiving team conflicts as a main cause of pressure in the job predicted higher burnout. Conclusions — Simple open questions coupled with quantitative measures appear a promising tool to investigate morale of mental health professionals and identify factors determining morale. Research, training and service development should focus on relationship aspects both with patients and within teams to reduce burnout in CMHCs.

Declaration of Interest

G.M.G. was employed by Modena local mental health service, but did not work in CMHCs. No other conflict of interest.

The diffusion and the reason for the use of complementary and alternative medicine among users of mental health services: a systematic review of literature

Aims- A systematic review of the studies on use of alternative and complementary medicine by mental health services' users to investigate the prevalence and motivations of CAM used to treat mental disorders.Methods- This paper shows a part of the results of a literature review, limited to studies between 1966 and 2005, was carried out using Ovid Medline, PsycINFO and EBM Reviews databases. This was supplemented by hand searching of reference lists.Results- Ninety-one studies on CAM's use by psychiatric patients were analyzed. In this paper the twenty-five articles about the CAM's prevalence and use's motivations are described. The use of complementary and alternative medicine (CAM) varies across countries: data suggest that from 50% to 80% of people suffering of mental disorders use CAM. Heterogeneous methods, samples and results characterizing the studies make generalizations results difficult. Patients motivations of CAM's use were poorly investigated by research, however it seems that persons affected by mental disorders use MAC prevalently because they are dissatisfied with conventional medicine and/or because these alternatives approaches are congruent with their own values, beliefs and philosophical orientation toward health care.Conclusions- The review emphasize the importance of a best physicians' education and information about CAM to improve the management of non-conventional and orthodox medicine relationship. The motivations of CAM's use underline the relevance of patient's holistic conception and the therapeutic role of a patient-physician relationship based on emphaty. Besides it seems important to investigate furtherly patients' psychological and social aspects that influence their choise to use CAM.

Applications and usefulness of routine measurement of patients' satisfaction with community-based mental health care

OBJECTIVE

This study was conducted within the framework of the South-Verona Outcome Project (SVOP) in a sample of patients, with the full spectrum of psychiatric diagnoses, who are attending the South-Verona Community-based Mental Health Service (CMHS). It aims to exemplify some applications of routine measurements of service satisfaction and specifically: i) identify strengths and weaknesses, in the patients' perspectives, of a 'real world' service; ii) monitor whether this specific service provides satisfactory care over 3 years and iii) identify if there are any patients' characteristics that might be associated with service dissatisfaction.

METHOD

Service satisfaction was measured with the Verona Service Satisfaction Scale (VSSS) across three subsequent waves of the SVOP; frequency distributions of scores in the various VSSS domains were compared. The relationship between satisfaction with psychiatric care and a number of well-established mental health indicators, including socio-demographic, clinical and service intervention variables, was investigated using random-effect models.

RESULTS

The organization of service and the behaviour and manners of the professionals were the main service strengths. Weaknesses were identified in the physical layout of facilities, in the lack of involvement of relatives and in the information provided. Overall satisfaction was medium-high, while subjects with longer duration of service contact and higher disability were the most dissatisfied. The predictors we examined, however, explained only modest percentages of variance.

CONCLUSION

Repeated, routine assessments of service satisfaction have provided a clear view of the South-Verona CMHS' strengths and weaknesses; this set of information was crucial for the continuous quality improvement process in the service.

Predictors of changes in caregiving burden in people with schizophrenia: a 3-year follow-up study in a community mental health service

OBJECTIVE

The aims of this study were to measure changes over 3-years in caregiving burden and emotional distress in relatives of people with schizophrenia and to identify factors predicting the levels of caregiving burden.

METHOD

A cohort of 51 caregivers of patients with schizophrenia attending the South-Verona Community Mental Health Service was assessed over 3-years with the Involvement Evaluation Questionnaire, European Version. Predictors of caregiving burden included both caregivers' and patients' characteristics and patterns of carer-patient interaction.

RESULTS

Baseline levels of family burden were high in worrying and urging domains. Fifty-one per cent of caregivers experienced significant emotional distress. Both overall burden and emotional distress improved. Higher patients' psychopathology, higher numbers of patient-rated needs, patients' lower global functioning and patients' poorer quality of life were found to be related to the severity of family burden. The only significant predictor of caregivers' burden at follow-up was the baseline level of caregivers' burden itself.

CONCLUSION

A policy addressing the caring burden of informal caregivers beyond patients' symptoms reduction should be considered.

Outcome of patients dropping out from community-based mental health care: a 6-year multiwave follow-up study

OBJECTIVE

To test three hypotheses: i) is dropping out of community mental health care due to dissatisfaction? ii) Do those who appropriately interrupt mental health care need any more help? iii) Do those who need treatment continue to receive it?

METHOD

A cohort from a Community Mental Health Service in Verona, Italy, was followed up for 6 years, interviewing patients at follow-up who are both in contact and not in contact with the service.

RESULTS

The main reason for dropping out was service dissatisfaction. Patients still in contact with the service were more often psychotic. Many patients no longer in contact had mild-to-moderate problems, especially anxiety and depression and some social disability. Patients out of contact rarely sought help from other agencies.

CONCLUSION

Comprehensive community mental health services seem to offer good continuity of mental health care to patients with psychosis, but dedicate less attention to patients with less severe problems. Some of these patients will go on experiencing a burden of symptoms and disability, over the years.

Predicting clinical and social outcome of patients attending 'real world' mental health services: a 6-year multi-wave follow-up study

OBJECTIVE

The aims of this study were: i) to determine changes in symptoms and social disability, and ii) to explore predictors of clinical and social outcome in patients receiving community-based mental health care.

METHOD

A total of 354 patients treated in the South-Verona Community Mental Health Service were followed up over 6 years (with assessments made at baseline, 2 and 6 years) by using a set of standardized measures exploring psychopathology (Brief Psychiatric Rating Scale) and social disability (Disability Assessment Schedule). Generalized linear latent and mixed models were used to explore longitudinal predictors of clinical and social outcome.

RESULTS

Psychotic patients displayed clinical and social outcome characterized by complex patterns of exacerbation and remission over time; however, a clear trend towards a deteriorating course was not found, thus challenging the notion that psychotics are fatally prone to a destiny of chronicity. Non-psychotics reported a significant reduction in the core symptom of depression and in the observable physical and motor manifestations of tension and agitation, and a parallel increase in complaints about their physical health. Clinical and social dimensions of outcome are influenced by specific and different set of predictors.

CONCLUSION

The results of this study confirm the need to implement naturalistic outcome studies conducted in the 'real world' services to inform decisions and strategies to be adopted in routine clinical practice.

Predictors of changes in needs for care in patients receiving community psychiatric treatment: a 4-year follow-up study

OBJECTIVE

This study aimed to investigate changes and predictors of change in needs for care, as assessed by both patients and mental health professionals, in a sample of subjects receiving community-based psychiatric care.

METHOD

The study was conducted using a 4-year prospective longitudinal design. A cohort of patients from the South-Verona Community Mental Health Service (CMHS) was assessed at baseline and follow-up using the Camberwell Assessment of Need, both staff and patient versions. Predictors of changes in needs were explored using block-stratified multiple regression analyses.

RESULTS

An overall stability for both patient-rated and staff-rated needs was found over time; however, significant changes in some specific need domains were found, such as self-rated health needs (improvement), self-rated social needs (deterioration) and staff-rated health needs (deterioration). Changes over time in self-rated and staff-rated needs are influenced by different and specific set of predictors, thus indicating that the two measures are not overlapping and convey different types of information.

CONCLUSION

Our data support the adoption of a negotiated approach in which both staff and users' views should be given equal weight when planning and providing needs-led mental health care.

Assessing the outcome of community-based psychiatric care: building a feedback loop from 'real world' health services research into clinical practice

OBJECTIVE

To describe the main characteristics of the South-Verona Outcome Project (SVOP) and to focus on its overall conceptual framework, with specific reference to the following perspectives: i) integrating evidence-based and practice-based approaches; ii) involving service professionals in routine outcome assessment; and iii) involving service users in mental health outcome assessment.

METHOD

A selective literature review of methodological and empirical papers addressing the relevance and usefulness of outcome research to routine clinical practice was performed.

RESULTS

Reviewed literature shows the need to integrate evidence-based and practice-based approaches and to involve service professionals in routine outcome assessment, by adopting a multiple perspective paradigm. Studies conducted in 'real world' health services indicate that the outcome of care is multifaceted and it can be perceived differently when different perspectives are taken into account. Such a complex picture can provide more comprehensive information on the effectiveness of care provided, to feed back positively into clinical practice.

CONCLUSION

The SVOP design and its methodological background were demonstrated to be appropriate for a detailed and routine assessment of outcome in the 'real world' of mental health services.

Do adults in contact with Australia's public sector mental health services get better?

This paper describes the outcomes of episodes of care for adults in public sector mental health services across Australia, with a view to informing the debate on service quality. Health of the Nation Outcome Scales (HoNOS) change scores and effect sizes were calculated for 14,659 acute inpatient episodes and 23,692 community episodes. The results showed that people in contact with public sector mental health services generally do get better, although the magnitude of improvement depends on the setting and episode type. This confirmatory finding is particularly positive, given current community concerns about the quality and effectiveness of mental health services.

Does additional care provided by a consumer self-help group improve psychiatric outcome? A study in an Italian community-based psychiatric service

This study compares the two-year clinical and social outcome, the use of services and the direct costs of patients of the South-Verona Community Psychiatric Service who were members of a self-help group, with those who were not. Use of services and costs in the two years before the baseline were compared with those occurring two years after the baseline. Self-help subjects decreased their use of hospital stay as to number of admissions and days in hospital, with a reduction of costs; they were more satisfied as to work/education while non self-help matches presented an increase of unmet needs. Clinical and social outcome showed no significant difference. The findings suggest that consumer participation may possibly enhance the effects of psychiatric treatment on outcome.

Changes and predictors of change in objective and subjective quality of life: multiwave follow-up study in community psychiatric practice

BACKGROUND

There is little knowledge of the predictors of objective and subjective quality of life.

AIMS

To describe changes at 2 and 6 years in objective and subjective quality of life in 261 individuals attending a community mental health service and to identify predictors of change in each life domain.

METHOD

Prospective study of demographic, diagnostic and service utilisation characteristics, psychopathology, functioning, disability, self-esteem, affect balance and service satisfaction.

RESULTS

Female gender, unmarried status, older age, less education and greater disability predicted a worsening of objective quality of life over time, but explain a small amount of variance. The variance in subjective quality of life was higher (>40%). Greater clinician-rated anxiety and depressive symptoms had a negative effect on satisfaction with health and general well-being. Psychological status, self-esteem and satisfaction with service were the most important predictors in almost all subjective domains; these variables should be important targets for treatment.

CONCLUSIONS

This study is the first to provide information to enable clinicians to make prognostic judgements about quality of life and plan effective therapeutic strategies to improve quality of life.

Routine measurement of outcomes in Australia's public sector mental health services

Objective

This paper describes the Australian experience to date with a national 'roll out' of routine outcome measurement in public sector mental health services.

Methods

Consultations were held with 123 stakeholders representing a range of roles.

Results

Australia has made an impressive start to nationally implementing routine outcome measurement in mental health services, although it still has a long way to go. All States/Territories have established data collection systems, although some are more streamlined than others. Significant numbers of clinicians and managers have been trained in the use of routine outcome measures, and thought is now being given to ongoing training strategies. Outcome measurement is now occurring 'on the ground'; all States/Territories will be reporting data for 2003–04, and a number have been doing so for several years. Having said this, there is considerable variability regarding data coverage, completeness and compliance. Some States/Territories have gone to considerable lengths to 'embed' outcome measurement in day-to-day practice. To date, reporting of outcome data has largely been limited to reports profiling individual consumers and/or aggregate reports that focus on compliance and data quality issues, although a few States/Territories have begun to turn their attention to producing aggregate reports of consumers by clinician, team or service.

Conclusion

Routine outcome measurement is possible if it is supported by a co-ordinated, strategic approach and strong leadership, and there is commitment from clinicians and managers. The Australian experience can provide lessons for other countries.