Strengths and Difficulties Questionnaire Added Value Scores: evaluating effectiveness in child mental health interventions

Minimally important change on the Columbia Impairment Scale and Strengths and Difficulties Questionnaire in youths seeking mental healthcare

BACKGROUND

Evidence-based mental health requires patient-relevant outcome data, but many indicators lack clinical meaning and fail to consider youth perceptions. The minimally important change (MIC) indicator designates change as meaningful to patients, yet is rarely reported in youth mental health trials.

OBJECTIVE

This study aimed to establish MIC thresholds for two patient-reported outcome measures (PROMs), the Columbia Impairment Scale (CIS) and the Strengths and Difficulties Questionnaire (SDQ), using different estimation methods.

METHODS

A sample of 247 youths (14-17 years) completed the CIS and SDQ at baseline and at 6 months in a youth mental health and substance use trial. At 6 months, youths also reported perceived change. Three anchor-based (mean change, receiver operating characteristic analysis, predictive modelling) and three distribution-based methods (0.5 SD, measurement error, smallest detectable change) were compared.

FINDINGS

Different methods yielded varying MIC thresholds. Predictive modelling provided the most precise anchor-based MIC: -2.6 points (95% CI -3.6, -1.6) for the CIS and -1.7 points (95% CI -2.2, -1.2) for the SDQ, indicating that score improvements of 12% for the CIS and 8% for the SDQ may be perceived as 'important' by youths. However, correlations between change score and anchor were below 0.5 for both measures, indicating suboptimal anchor credibility. Stronger correlations between the anchor and T2 PROM scores compared with T1 scores suggest the presence of recall bias. All MIC estimates were smaller than the smallest detectable change.

CONCLUSIONS

Predictive modelling offers the most precise MIC, but limited anchor credibility suggests careful anchor calibration is necessary.

CLINICAL IMPLICATIONS

Clinicians may consider the MIC CI as indicative of meaningful change when discussing treatment impact with patients.

TRIAL REGISTRATION NUMBER

NCT02836080.

Outcomes for Adaptive Mentalization Based Integrative Treatment informed care for adolescents using a deployment-based approach

PURPOSE

Adolescent mentalisation-based integrative therapy (AMBIT) is a whole-systems approach designed to enhance the effectiveness and coordination of care for clients experiencing severe and pervasive difficulties in social and health care settings, who have not responded to traditional clinical approaches. AMBIT is a team-based manualised method that primarily aims to bolster mental state understanding and discourse focused on the client within and between teams. Over 300 teams worldwide have been trained in and adhere to AMBIT principles.

METHOD

In this paper, we review and summarise the outcomes reported by eight AMBIT-informed teams that have published their findings with young people. Each report is discussed, and limitations of the data provided are identified.

RESULTS

A synthesis of the findings across the studies suggests a generally positive impact of teams informed by AMBIT with moderate to large effect sizes on reducing symptoms and improving functionality.

CONCLUSIONS

This study suggests that AMBIT may be a promising approach for young people with multiple problems but further research is needed to identify the active mechanisms of change in complex helping systems.

Trauma informed occupational therapy school practice with adolescents with social emotional and behavioural difficulties: findings of standardised measures

Abstract Introduction Although non-standardized qualitative and quantitative reports from 39 Irish students aged 12 to 14 years, with complex social and emotional needs and their 13 teachers had established benefits of ‘Movement Matters’, an occupational therapy designed participatory curriculum for learning self-regulation in mainstream schools (National Behaviour Support Service, 2015a), standardized measures were required for quality assurance by the Irish Government funded service. Objective The relevance of findings of standardized attitude measurements on the impacts of an occupation focused intervention with a discrete school population in areas of social disadvantage are discussed. Method Findings of the ‘Pupil Attitude to Self and School’ (PASS) (Granada Learning, 2021) and ‘Strengths and Difficulties Questionnaire’ (SDQ) (Youth in Mind, 2021), both well-established standardized measures of attitude and behaviours completed (pre and post intervention) by teachers, parents and students are presented. Results No statistically significant differences were found between the standardized pre and post intervention measures. However, findings indicate a disparity between how these students in complex environments experience school in comparison to large UK study populations. Conclusion This paper discusses the use of these standardized measures for capturing impacts of an occupation focused intervention in a school environment. It calls for an increased appreciation on outcomes measurement related to the intervention’s aim of providing experiences of meaningful participation and well-being. Further collaborative research on outcome measurement of participation of students in complex mainstream school environments is required.

A Systematic Review of Household and Family Alcohol Use and Childhood Neurodevelopmental Outcomes in Low- and Middle-Income Countries

Childhood exposure to alcohol misuse by household adults has been related to childhood developmental delay, cognitive impacts, mental illness, and problem behaviours. Most evidence comes from high income countries. This systematic review only included studies from low- and middle-income countries (LMICs). Five databases were searched from 1990-2020. Twenty-eight studies of children 0-12 years were included, with 42,599 participants from 11 LMICs. The most common outcome was behavioural problems/disorders (19 studies). Despite varying study designs, this review found that alcohol misuse by household members in LMICs is associated with adverse child neurodevelopmental outcomes, although casual inferences cannot be drawn in the absence of well conducted prospective studies. Statistically significant correlations were described between parental alcohol misuse and child emotional and behavioural difficulties, cognitive delay, and risky behaviours. In future, prospective cohort studies are recommended, with adjustment for confounders.

Cumulative risk exposure and emotional symptoms among early adolescent girls

Background

From early adolescence, girls and women report the highest rates of emotional symptoms, and there is evidence of increased prevalence in recent years. We investigate risk factors and cumulative risk exposure (CRE) in relation to emotional symptoms among early adolescent girls.

Methods

We used secondary data analysis, drawing on data capturing demographic information and self-reported emotional symptoms from 8327 girls aged 11–12 years from the 2017 baseline data collection phase of the HeadStart evaluation. We used structural equation modelling to identify risk factors in relation to self-reported emotional symptoms, and collated this into a CRE index to investigate associations between CRE and emotional symptoms.

Results

Four risk factors were found to have a statistically significant relationship with emotional symptoms among early adolescent girls: low academic attainment, special educational needs, low family income, and caregiving responsibilities. CRE was positively associated with emotional symptoms, with a small effect size.

Conclusions

Results identify risk factors (outlined above) that are associated with emotional symptoms among early adolescent girls, and highlight that early adolescent girls experiencing a greater number of risk factors in their lives are likely to also experience greater emotional distress. Findings highlight the need for identification and targeted mental health intervention (e.g., individual or group counselling, approaches targeting specific symptoms), for those facing greater risk and/or with emergent symptoms.

Australian Maltreated Infants and Young Children Can Achieve Positive Relational Health With Neurodevelopmentally- and Trauma-Informed Interventions Provided Within Relationally-Positive and Stable Environments

Background: Childhood maltreatment such as abuse, neglect and family violence has a profound impact on children's psychological and relational functioning and their lifelong trajectory, with associated adverse physical and mental health outcomes, higher mortality rates and reduced socioeconomic opportunities. The aim of the study was to explore the impact of neurodevelopmentally- and trauma-informed interventions on the relational health of children who have experienced maltreatment. Context: The study was conducted at Berry Street Take Two, an Australian therapeutic service. Take Two provides services to Victorian children aged 0-18 years, to address the impact of the trauma they have experienced from maltreatment. Take Two clinicians use relational and ecological frameworks, neurodevelopmental research and evidence-informed approaches to repair family relationships and develop networks of caring adults that focus on meeting the child's needs. Take Two uses the NMT approach as a framework for clinical intervention-planning and is site-certified in the use of the NMT Clinical Practice tools. Method: The mixed methods study had two components. A cross sectional study of baseline and repeat clinical measure data (HoNOSCA and SDQ) with a cohort of children aged 2-11 years (n = 91), who were clients of Berry Street Take Two between 2014 and 2019, was conducted utilizing SPSS. The quantitative data analysis was supplemented by three case studies of Berry Street Take Two clients, which explored the process of intervention, including intervention type, timing and dosage. The case studies drew on the full case record for each child to illustrate the impact of NMT-informed interventions on the relational health, psychological and behavioral functioning of children. Results: The study found that Take Two intervention was associated with improved relational health, measured by the NMT metric and supported by significant positive changes on the SDQ and HoNOSCA with medium effect sizes (cohen's d). The case study analysis highlighted the importance of intervention addressing individual, family and systems elements to bring about positive change. Conclusions: This study illustrates the value of neurodevelopmental trauma-informed interventions in positively impacting on the relational health and current functioning of maltreated children and the potential to reduce the lifelong impact of maltreatment.

Virtual controls as an alternative to randomized controlled trials for assessing efficacy of interventions

Randomized controlled trials are ubiquitously spoken of as the "gold standard" for testing interventions and establishing causal relations. This article presents evidence for two premises. First: there are often major problems with randomized designs; it is by no means true that the only good design is a randomized design. Second: the method of virtual controls in some circumstances can and should replace randomized designs.Randomized trials can present problems with external validity or generalizability; they can be unethical; they typically involve much time, effort, and expense; their assignments to treatment conditions often can be maintained only for limited time periods; examination of their track record reveals problems with reproducibility on the one hand, and lack of overwhelming superiority to observational methods on the other hand.The method of virtual controls involves ongoing efforts to refine statistical models for prediction of outcomes from measurable variables, under conditions of no treatment or current standard of care. Research participants then join a single-arm study of a new intervention. Each participant's data, together with the formulas previously generated, predict that participant's outcome without the new intervention. These outcomes are the "virtual controls." The actual outcomes with intervention are compared with the virtual control outcomes to estimate effect sizes. Part of the research product is the prediction equations themselves, so that in clinical practice, individual treatment decisions may be aided by quantitative answers to the questions, "What is estimated to happen to this particular patient with and without this treatment?"The method of virtual controls is especially indicated when rapid results are of high priority, when withholding intervention is likely harmful, when adequate data exist for prediction of untreated or standard of care outcomes, when we want to let people choose the treatment they prefer, when tailoring treatment decisions to individuals is desirable, and when real-world clinical information can be harnessed for analysis.

Generalized linear mixed-model (GLMM) trees: A flexible decision-tree method for multilevel and longitudinal data

Objective: Decision-tree methods are machine-learning methods which provide results that are relatively easy to interpret and apply by human decision makers. The resulting decision trees show how baseline patient characteristics can be combined to predict treatment outcomes for individual patients, for example. This paper introduces GLMM trees, a decision-tree method for multilevel and longitudinal data. Method: To illustrate, we apply GLMM trees to a dataset of 3,256 young people (mean age 11.33, 48% girls) receiving treatment at one of several mental-health service providers in the UK. Two treatment outcomes (mental-health difficulties scores corrected for baseline) were regressed on 18 demographic, case and severity characteristics at baseline. We compared the performance of GLMM trees with that of traditional GLMMs and random forests. Results: GLMM trees yielded modest predictive accuracy, with cross-validated multiple R values of .18 and .25. Predictive accuracy did not differ significantly from that of traditional GLMMs and random forests, while GLMM trees required evaluation of a lower number of variables. Conclusion: GLMM trees provide a useful data-analytic tool for clinical prediction problems. The supplemental material provides a tutorial for replicating the GLMM tree analyses in R.

Fathering Through Change (FTC) intervention for single fathers: Preventing coercive parenting and child problem behaviors

Dishion and Patterson's work on the unique role of fathers in the coercive family process showed that fathers' coercion explained twice the variance of mothers' in predicting children's antisocial behavior and how treatment and prevention of coercion and promotion of prosocial parenting can mitigate children's problem behaviors. Using these ideas, we employed a sample of 426 divorced or separated fathers randomly assigned to Fathering Through Change (FTC), an interactive online behavioral parent training program or to a waitlist control. Participating fathers had been separated or divorced within the past 24 months with children ages 4 to 12 years. We tested an intent to treat (ITT) mediation hypothesis positing that intervention-induced changes in child problem behaviors would be mediated by changes in fathers' coercive parenting. We also tested complier average causal effects (CACE) models to estimate intervention effects, accounting for compliers and noncompliers in the treatment group and would-be compliers in the controls. Mediation was supported. ITT analyses showed the FTC obtained a small direct effect on father-reported pre-post changes in child adjustment problems (d = .20), a medium effect on pre-post changes in fathers' coercive parenting (d = .61), and a moderate indirect effect to changes in child adjustment (d = .30). Larger effects were observed in CACE analyses.

Application of the Strengths and Difficulties Questionnaire Added Value Score in evaluating the effectiveness of Functional Family Therapy within local authority social work services

BACKGROUND

As 'Blueprints' evidence-based programmes, such as Functional Family Therapy (FFT), originating from the United States, are increasingly implemented in Social Work services, the importance of assessing their effectiveness in a UK context is crucial. To do this, it is not always practical for services to commission randomised control trials or quasi-experimental control trials. The Strengths and Difficulties Questionnaire (SDQ) Added Value Score has been shown to have utility in the evaluation of intervention programmes by controlling for regression to the mean, attenuation and the shifting nature of most childhood psychopathology.

METHOD

The SDQ Added Value Score was used to assess the effectiveness of FFT in two local authorities in Scotland. One hundred and sixty-four families who had finished FFT completed the Strengths and Difficulties Questionnaire, the Outcome Questionnaire and the Client Outcome Measure at pre- and postintervention.

RESULTS

Both parents' and adolescents' average psychosocial distress scores significantly decreased on all measures after FFT and many of the scores postintervention fell to a range equivalent with the general population. Furthermore, calculation of the SDQ Added Value Score indicated that adolescents' mean total difficulties scores were lower following FFT than what would have been expected had this intervention not been received, producing an effect size that compares favourably to other interventions.

CONCLUSIONS

Functional Family Therapy has been identified as an effective intervention for improving the psychosocial functioning of high-risk adolescents and their families.

Review of the strengths and difficulties questionnaire translated into languages spoken by children and adolescents of refugee background

The Strengths and Difficulties Questionnaire (SDQ), routinely used to screen for children's and adolescents' emotional and behavioural problems, has been translated into at least 80 languages. As children of refugee background are particularly vulnerable to mental health problems resulting from their refugee experiences, this review examines whether SDQs translated into languages spoken by major refugee groups are validated and culturally equivalent to the original SDQ and sensitive to change following interventions. No reviewed studies of translated SDQs reported on challenges in achieving conceptual and linguistic equivalence in translation. Factor analysis predominantly showed structural inequivalence with the original 5-factor model, suggesting translated SDQ subscales may measure different constructs. Predictive equivalence findings tended to show somewhat higher sensitivity for detecting disorder than the original SDQ's low sensitivity, and somewhat lower specificity. Outcome studies yielded equivocal results with refugee and immigrant groups. SDQ items do not detect the psychological sequelae of trauma; hence it is recommended that the SDQ be used with caution to screen refugee children, with a follow-up clinical interview for verification. Cross-cultural qualitative research is needed into parents' and adolescents' interpretation of translated SDQ items.

Associations between family and clinician ratings of child mental health: A study of UK CAMHS assessments and outcomes

BACKGROUND

The rated severity of child mental health problems depends on who is doing the rating, whether child, carer or clinician. It is important to know how these ratings relate to each other.

AIMS

To investigate to what extent clinicians' views are associated with carers' and young people's views in routine care in the United Kingdom.

METHOD

Ratings of clinician and parent/child viewpoints from a large Child and Adolescent Mental Health Services (CAMHS) sample ( ns 1773-47,299), as measured by the Children's Global Assessment Scale (CGAS) and Strengths and Difficulties Questionnaire (SDQ) respectively, were analysed. The parent SDQ added value score (AVS), which adjusts for regression to the mean and other non-treatment change, was also included in the analyses.

RESULTS

Small-to-medium correlations were found between family and clinician ratings; however, ratings diverged for the lowest-function CGAS bands. Regression analyses showed that pro-social ratings from both child and parent contributed to clinician ratings. Knowing child-reported emotional problem severity made parent ratings of emotions irrelevant to clinician judgements. There was a positive association between SDQ AVS and CGAS; as hypothesised, CGAS showed more change than the SDQ AVS, suggesting that clinicians over-estimate change.

CONCLUSION

This study shows the importance of multi-informant data gathering and the integration of multiple views by clinicians when monitoring outcomes.

Service-level variation, patient-level factors, and treatment outcome in those seen by child mental health services

Service comparison is a policy priority but is not without controversy. This paper aims to investigate the amount of service-level variation in outcomes in child mental health, whether it differed when examining outcomes unadjusted vs. adjusted for expected change over time, and which patient-level characteristics were associated with the difference observed between services. Multilevel regressions were used on N = 3256 young people (53% male, mean age 11.33 years) from 13 child mental health services. Outcome was measured using the parent-reported Strengths and Difficulties Questionnaire. The results showed there was 4-5% service-level variation in outcomes. Findings were broadly consistent across unadjusted vs. adjusted outcomes. Young people with autism or infrequent case characteristics (e.g., substance misuse) had greater risk of poor outcomes. Comparison of services with high proportions of young people with autism or infrequent case characteristics requiring specialist input needs particular caution as these young people may be at greater risk of poor outcomes.

Caregiver alcohol use and mental health among children orphaned by HIV/AIDS in South Africa

Research in the developed world suggests that parental alcohol use negatively impacts child mental health. However, little research has examined these relations among children in the developing world and no studies to date have done so in the context of AIDS-orphanhood. Therefore, the present study tested the interactive effect of AIDS-orphan status with caregiver alcohol use on child mental health. The sample included 742 children (51.2% female; Mage = 9.18; age range: 7-11 years; 29.8 AIDS-orphans; 36.8% orphaned by causes other than AIDS; 33.4% non-orphaned) recruited from Mangaung in the Free State Province of South Africa. Child mental health was assessed via child self-report, caregiver, and teacher reports; and caregiver alcohol use via self-report. Path analyses, via structural equation modeling, revealed significant direct effects for AIDS-orphan status on caregiver-reported child mental health; and for caregiver alcohol-use problems on teacher-reported child mental health. However, the interaction effect of AIDS-orphan status with caregiver alcohol use did not reach significance on all three reports of child mental health problems. These results suggest that orphan status and caregiver alcohol use may independently relate to mental health problems in children and that the effects of both should be considered in the context of the mental health needs of children in AIDS-affected countries.

Routine mental health outcome measurement in the UK

This paper offers a short history of routine clinical outcomes measurement (RCOM) in UK mental health services. RCOM developments in primary and secondary care are described, with reference to measures currently in widespread use or likely to be implemented. Assessment procedure and completion rates are discussed. Some of the forces operating in this field are enumerated. Comparison is made with UK attempts at routine outcomes measurement in public education. This field is thus reviewed for lessons for RCOM, and opportunities and challenges considered.

Pre-school screening for developmental and emotional health: Comparison with neurodevelopmental assessment

AIM

The study aim was to compare detection of and referral for developmental and emotional problems in a school readiness screening programme (New Zealand Before School Check, B4SC) with that of a comprehensive neurodevelopmental assessment.

METHODS

This is a prospective cohort study of children (n = 274) born at risk of neonatal hypoglycaemia and recruited to a follow-up study of neurodevelopmental outcomes at 4.5 years (Children with Hypoglycaemia and their Later Development (CHYLD) Study). Children identified as of significant concern for developmental and emotional problems, and referrals made, were compared in the B4SC and CHYLD Study. Scores of the parent-completed Strengths and Difficulties Questionnaire used in both assessments were compared.

RESULTS

Of the 274 children who underwent clinical neurodevelopmental assessment at a mean (standard deviation) age of 53.3 (1.8) months, 237 had the B4SC developmental and emotional health screening. Of these, 44 (19%) children met B4SC referral criteria, and 15 (6%) were referred, but only 21 (9%) children met CHYLD referral criteria, and 10 (4%) were referred. Twelve children (5%) met both the B4SC and CHYLD referral criteria, and two were referred by both. When assessed twice, 39 (17%) children changed parent-completed Strengths and Difficulties Questionnaire category. Children who did not have B4SC screening had higher mean total difficulties score (10.5 vs. 8.2, P = 0.009) and were more likely to have cognitive delay than those who were screened (19% vs. 8%, P = 0.04).

CONCLUSION

More children met referral criteria for the B4SC screening programme than for a more comprehensive neurodevelopmental assessment. Children who did not have screening had a higher incidence of cognitive and behaviour problems than those who did.

Associations between evidence-based practice and mental health outcomes in child and adolescent mental health services

The effectiveness of evidence-based practice in the treatment of children with conduct disorder (n = 186) or emotional disorders (n = 490) in routine care was examined using naturalistic, previously collected data from 30 child and adolescent mental health services. Repeated measures analysis of covariance was used to compare the outcomes of children who received parent training for conduct disorder and cognitive behavioural therapy for emotional disorders (evidence-based practice) with children who did not receive these treatments (non-evidence-based practice). There was a relatively low occurrence of evidence-based practice, particularly for children with conduct disorder. Both the evidence-based practice and non-evidence-based practice groups improve over time, with moderate effect sizes, and there were greater improvements associated with evidence-based practice for children with emotional disorders, based on child self-reported symptoms but not on parent report. In the present sample, significant differences were not found for conduct disorder. Findings provide tentative support for evidence-based practice for the treatment of emotional disorders in routine care settings.

Analysing and reporting UK CAMHS outcomes: an application of funnel plots

BACKGROUND

Patient-reported outcomes measures are increasingly being used in child and adolescent mental health services (CAMHS). League tables are a common way of comparing organizations across health and education but have limitations that are not well known in CAMHS.

METHOD

Parent-rated Strengths and Difficulties Questionnaire (SDQ) outcomes data from 15,771 episodes of care across 51 UK CAMHS were analysed using funnel plots, an alternative to league tables.

RESULTS

While most services were indistinguishable from the national average there was evidence of heterogeneous outcomes and seven services had outcomes below 99.9% limits for SDQ added-value scores.

CONCLUSIONS

Funnel plots are powerful tools for navigating national data and can help prompt investigations using clinical theory and local service context. Examples are provided of factors to consider in these investigations. We argue that analyses of the local context are central to the valid application of funnel plots.

Occupied bed days a redundant currency? An evaluation of the first 10 years of an integrated model of care for mentally ill adolescents

BACKGROUND

We report on clinical outcomes from the first 10 years of operation of an Adolescent Assertive Outreach Team (AAOT) which forms part of an integrated Tier 4 service offering inpatient, day-patient and intensive community treatment.

METHOD

Prospective data were recorded at pre- and post-treatment for patients accepted for treatment by the AAOT over a 10-year period.

RESULTS

A total of 393 adolescents were treated by the AAOT, and 135 received some hospital care as part of their treatment. Contact periods with the team were significantly shorter for those treated wholly in the community. Outcome measures indicated severe pathology at onset with substantial improvement in function regardless of whether treatment involved hospital care or community support only.

CONCLUSIONS

Adolescent Outreach is a viable and sustainable alternative to inpatient care for some young people referred for admission, reducing but not eliminating the need for acute beds.

Comparison of indices of clinically meaningful change in child and adolescent mental health services: difference scores, reliable change, crossing clinical thresholds and 'added value' - an exploration using parent rated scores on the SDQ

BACKGROUND

Establishing what constitutes clinically significant change is important both for reviewing the function of services and for reflecting on individual clinical practice. A range of methods for assessing change exist, but it remains unclear which are best to use and under which circumstances.

METHOD

This paper reviews four indices of change [difference scores (DS), crossing clinical threshold (CCT), reliable change index (RCI) and added value scores (AVS)] drawing on outcome data for 9764 young people from child and adolescent mental health services across England.

RESULTS

Looking at DS, the t-test for time one to time two scores indicated a significant difference between baseline and follow up scores, with a standardised effect size of d = 0.40. AVS analysis resulted in a smaller effect size of 0.12. Analysis of those crossing the clinical threshold showed 21.2% of cases were classified as recovered, while 5.5% were classified as deteriorated. RCI identified 16.5% of cases as showing reliable improvement and 2.3% of cases as showing reliable deterioration. Across RCI and CCT 80.5% of the pairings were exact (i.e., identified in the same category using each method).

CONCLUSIONS

Findings indicate that the level of agreement across approaches is at least moderate; however, the estimated extent of change varied to some extent based on the index used. Each index may be appropriate for different contexts: CCT and RCI may be best suited to use for individual case review; whereas DS and AVS may be more appropriate for case-mix adjusted national reporting.

Uses and abuses of patient reported outcome measures (PROMs): potential iatrogenic impact of PROMs implementation and how it can be mitigated

Having been a national advocate for the use of patient reported outcome measures (PROMs) in Child and Adolescent Mental Health Services (CAMHS) in the UK for the last decade, I have become increasingly concerned that unless the potential iatrogenic impact of widespread policy requirement for use of PROMs (Department of Health, Children and Young People’s Health Outcomes Strategy, 2012) is recognised and addressed their real potential benefits (Sapyta et al., J Clin Psychol 61(2):145–153, 2005) may never be realized. Drawing on examples from PROMs implementation in CAMHS in the UK (Wolpert et al., J Ment Health 21(2):165–173, 2012a; Child Adolesc Mental Health 17(3):129–130, 2012b). I suggest key ways forward if PROMs are to support best clinical practice rather than undermine it.

Measurement Issues: Review of four patient reported outcome measures: SDQ, RCADS, C/ORS and GBO - their strengths and limitations for clinical use and service evaluation

BACKGROUND

There is an international drive for routine use of Patient Reported Outcome Measures (PROMs) across all health services including in relation to Child and Adolescent Mental Health Services (CAMHS). A number of reviews have summarized the validity and reliability of well-being and mental health measures for children but there are fewer attempts to consider utility for routine use.

METHOD

This review considers four child self-report measures: the Strengths and Difficulties Questionnaire (SDQ), the Revised Child Anxiety and Depression Scale (RCADS), (Child) Outcomes Rating Scale (C/ORS) and Goals Based Outcomes (GBOs). It explores the strengths and limitations of each and considers how they can be used to support both clinical practice and service evaluation.

RESULTS

There is evidence for the clinical utility of RCADS, C/ORS and GBOs, although the utility of the SDQ as a feedback measure remains unclear. For service evaluation, the SDQ has the greatest evidence for norms making it useful for comparison and there is evidence that the RCADS may be the most sensitive to change of the measures reviewed; C/ORS has issues around ceiling effect, data error and data manipulation. More research is required around GBOs before their use for service evaluation can be determined.

CONCLUSIONS

In summary, these different measures may be viewed as complementary tools and determining the best way to make use of them severally and individually in clinical and community settings is a current focus for child mental health practitioners.

Innovations in Practice: Further evidence on the effectiveness of the strengths and difficulties added value score as an outcome measure for child and adolescent services

BACKGROUND

The Strengths and Difficulties Added Value Score (SDQ AVS) uses a large epidemiological study to predict follow-up parental SDQ scores for the evaluation of routine outcomes.

METHOD

We tested the prediction of the SDQ AVS derived from a national population survey separately on scores for the waiting list control and intervention groups in a randomised controlled trial. If the SDQ AVS is to be clinically useful, it needs to function as expected across different populations.

RESULTS

In the control arm, the SDQ AVS predicted an effect size of 0.15 (95% CI -0.01-0.30) compared to an expected effect size of 0, as the children in this arm received no treatment. In the experimental arm, the SDQ AVS predicted an effect size of 0.62 (95% CI 0.42-0.83) compared to the study effect size of 0.53. Change scores overestimated the effect size in both arms (control 0.50 95% CI 0.34-0.66, intervention 0.85 95% CI 0.66-1.04).

CONCLUSION

Our findings suggest that the SDQ AVS adjusts for spontaneous improvement, regression to the mean and attenuation.

The value of measuring impact alongside symptoms in children and adolescents: a longitudinal assessment in a community sample

The impact that psychiatric symptoms have on the lives of young people is central to clinical practice and classification. However, there is relatively little research on impact and its association with symptoms. This paper examines how well impact can be measured and how it relates to psychiatric outcomes. On four separate occasions over 3 years, symptoms and impact were assessed in a UK epidemiological sample (n = 4,479; 51.5 % boys) using the Strengths and Difficulties Questionnaire (SDQ) as reported by parents, youths and teachers. Disorders were ascertained using the Development and Well-Being Assessment. An impact scale made of items about distress and impairment demonstrated considerable internal consistency, cross-informant correlations, and longitudinal stability by all reporting sources. Impact at baseline was a strong predictor of psychiatric disorder 3 years later after accounting for psychiatric disorders and symptoms measured at baseline: odds ratio OR = 2.10, 95 % Confidence Interval (CI) [1.50, 2.94] according to parent-rated impact and OR = 1.71, CI [1.08, 2.72] according to teacher-rated impact. Changes in impact over time were predicted, but not fully accounted for, by symptoms measured at baseline. Impact can be reliably and easily measured across time, and it may be clinically useful as an independent predictor of future symptoms and psychiatric disorders. More studies are needed to understand inter-individual variation in the impact caused by equivalent symptoms.

Implementing routine outcome measures in child and adolescent mental health services: from present to future practice

BACKGROUND

Routine outcome measurement (ROM) in CAMHS is supported by U.K. Government policy. However, little is known about how measures are used in practice.

METHOD

Data describing use of ROM, knowledge and attitudes regarding implementation were collected using a regional case-note audit, online survey and stakeholder workshop.

RESULTS

While the principle of ROM was supported by stakeholders, baseline and follow-up outcome measurement occurred in less than a fifth of cases. Barriers to implementation included lack of training and resources, clinicians' perceptions of the limitations of existing measures and lack of regular feedback of outcome data.

CONCLUSIONS

Implementation of ROM may be facilitated by session-by-session measures with immediate feedback to clinicians and patients.

Patient-reported outcomes in child and adolescent mental health services (CAMHS): use of idiographic and standardized measures

BACKGROUND

There is increasing emphasis on use of patient-reported outcome measures (PROMs) in mental health but little research on the best approach, especially where there are multiple perspectives.

AIMS

To present emerging findings from both standardized and idiographic child-, parent- and clinician-rated outcomes in child and adolescent mental health services (CAMHS) and consider their correlations.

METHOD

Outcomes were collected in CAMHS across the UK. These comprised idiographic measures (goal-based outcomes) and standardized measures (practitioner-rated Children's Global Assessment Scale; child- and parent-rated Strengths and Difficulties Questionnaire).

RESULTS

There was reliable positive change from the beginning of treatment to later follow-up according to all informants. Standardized clinician function report was correlated with standardized child difficulty report (r  =  - 0.26), standardized parent report (r  =  - 0.28) and idiographic joint client-determined goals (r  =  0.38) in the expected directions.

CONCLUSIONS

These results suggest that routine outcome monitoring is feasible, and suggest the possibility of using jointly agreed idiographic measures alongside particular perspectives on outcome as part of a PROMs approach.

Reliability and sensitivity to change of the Family Life Questionnaire in a clinical population

BACKGROUND

  The Family Life Questionnaire (FLQ) is a new measure of family functioning, which acknowledges that the experience of the family unit may vary between different children. This study examined the reliability and sensitivity to change of the FLQ in a clinical population.

METHOD

  The FLQ was administered to 91 parents attending the first session of the Incredible Years parenting programmes in Devon during 2009, 71 of them completed it on a second occasion a week later and 55 on a third occasion at the end of the programme. Internal consistency, test-retest reliability and sensitivity to change were calculated.

RESULTS

  Internal consistency and test-retest reliability of the scales varied between moderate and very good, except for the discipline and special allowances items which had poor internal consistency when grouped as a scale. The measures showed promising evidence of sensitivity to change.

CONCLUSIONS

  The FLQ is a reliable measure of family function that seems sensitive to change. There is some evidence of validity but a lack of suitable comparators limited this part of the study. The study sample was too small to explore the scale structure.

Mental health and childhood adversities: a longitudinal study in Kabul, Afghanistan

OBJECTIVE

To identify prospective predictors of mental health in Kabul, Afghanistan.

METHOD

Using stratified random-sampling in schools, mental health and life events for 11-to 16-year-old students and their caregivers were assessed. In 2007, 1 year after baseline, the retention rate was 64% (n = 115 boys, 119 girls, 234 adults) with no evidence of selection bias. Self- and caregiver-rated child mental health (Strengths and Difficulties Questionnaire), depressive (Depression Self-Rating Scale), and posttraumatic stress (Child Revised Impact of Events Scale) symptoms and caregiver mental health (Self-Report Questionnaire) were assessed. Lifetime trauma and past-year traumatic, stressful, and protective experiences were assessed.

RESULTS

With the exception of posttraumatic stress, one-year trajectories for all mental health outcomes showed significant improvement (p < .001). Family violence had a striking impact on the Strengths and Difficulties Questionnaire data, raising caregiver-rated scores by 3.14 points (confidence interval [CI] 2.21-4.08) or half a standard deviation, and self-rated scores by 1.26 points (CI 0.50-2.03); past-year traumatic beatings independently raised self-rated scores by 1.85 points (CI 0.03-3.66). A major family conflict raised depression scores by 2.75 points (CI 0.89-4.61), two thirds of a standard deviation, whereas improved family life had protective effects. Posttraumatic stress symptom scores, however, were solely contingent on lifetime trauma, with more than three events raising scores by 5.38 points (CI 1.76-9.00).

CONCLUSIONS

Family violence predicted changes in mental health problems other than posttraumatic stress symptoms in a cohort that showed resilience to substantial socioeconomic and war-related stressors. The importance of prospectively identifying impacts of specific types of childhood adversities on mental health outcomes is highlighted to strengthen evidence on key modifiable factors for intervention in war-affected populations.

Admissions to acute adolescent psychiatric units: a prospective study of clinical severity and outcome

BACKGROUND

Several countries have established or are planning acute psychiatric in-patient services that accept around-the-clock emergency admission of adolescents. Our aim was to investigate the characteristics and clinical outcomes of a cohort of patients at four Norwegian units.

METHODS

We used a prospective pre-post observational design. Four units implemented a clinician-rated outcome measure, the Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA), which measures mental health problems and their severity. We collected also data about the diagnoses, suicidal problems, family situations, and the involvement of the Child Protection Service. Predictions of outcome (change in HoNOSCA total score) were analysed with a regression model.

RESULTS

The sample comprised 192 adolescents admitted during one year (response rate 87%). Mean age was 15.7 years (range 10-18) and 70% were girls. Fifty-eight per cent had suicidal problems at intake and the mean intake HoNOSCA total score was 18.5 (SD 6.4). The largest groups of main diagnostic conditions were affective (28%) and externalizing (26%) disorders. Diagnoses and other patient characteristics at intake did not differ between units. Clinical psychiatric disorders and developmental disorders were associated with severity (on HoNOSCA) at intake but not with outcome. Of adolescents ≥ 16 years, 33% were compulsorily admitted. Median length of stay was 8.5 days and 75% of patients stayed less than a month. Compulsory admissions and length of stay varied between units. Mean change (improvement) in the HoNOSCA total score was 5.1 (SD 6.2), with considerable variation between units. Mean discharge score was close to the often-reported outpatient level, and self-injury and emotional symptoms were the most reduced symptoms during the stay. In a regression model, unit, high HoNOSCA total score at intake, or involvement of the Child Protection Service predicted improvement during admission.

CONCLUSIONS

Acute psychiatric in-patient units for adolescents effectively meet important needs for young people with suicidal risks or other severe mental health problems. These units may act in suicide prevention, stabilizing symptom severity at a lower level within a short stay. It is important to explore the differences in outcome, compulsory admissions, and length of stay between units.