Investigating Clinical Excellence and Impact Awards (INCEA): a qualitative study into how current assessors and other key stakeholders define and score excellence

OBJECTIVES

The National Clinical Excellence Awards (NCEAs) in England and Wales were designed, as a form of performance-related pay, to reward high-performing senior doctors and dentists. To inform future scoring of applications and subsequent schemes, we sought to understand how current assessors and other stakeholders would define excellence, differentiate between levels of excellence and ensure unbiased definitions and scoring.

DESIGN

Semistructured qualitative interview study.

PARTICIPANTS

25 key informants were identified from Advisory Committee on Clinical Excellence Awards subcommittees, and relevant professional organisations in England and Wales. Informants were purposively sampled to achieve variety in gender and ethnicity.

FINDINGS

Participants reported that NCEAs had a role in incentivising doctors to strive for excellence. They were consistent in identifying 'clinical excellence' as involving making an exceptional difference to patients and the National Health Service, and in going over and above the expectations associated with the doctor's job plan. Informants who were assessors reported: encountering challenges with the current scoring scheme when seeking to ensure a fair assessment; recognising tendencies to score more or less leniently; and the potential for conscious or unconscious bias in assessments. Particular groups of doctors, including women, doctors in some specialties and settings, doctors from minority ethnic groups, and doctors who work less than full time, were described as being less likely to self-nominate, lacking support in making applications or lacking motivation to apply on account of a perceived likelihood of not being successful. Practical suggestions were made for improving support and training for applicants and assessors.

CONCLUSIONS

Participants in this qualitative study identified specific concerns in respect of the current approaches adopted in applying for and in assessing NCEAs, pointing to the importance of equity of opportunity to apply, the need for regular training for assessors, and to improved support for applicants and potential applicants.

Individual supported work placements (ReISE) for improving sustained return to work in unemployed people with persistent pain: study protocol for a cohort randomised controlled trial with embedded economic and process evaluations

BACKGROUND

Around one-third of workdays lost in Norway are due to musculoskeletal conditions, with persistent (chronic) pain being the most frequent cause of sick leave and work disability. Increasing work participation for people with persistent pain improves their health, quality of life, and well-being and reduces poverty; however, it is not clear how to best help unemployed people who have persistent pain to return to work. The aim of this study is to examine if a matched work placement intervention featuring case manager support and work-focused healthcare improves return to work rates and quality of life for unemployed people in Norway with persistent pain who want to work.

METHODS

We will use a cohort randomised controlled approach to test the effectiveness and cost-effectiveness of a matched work placement intervention featuring case manager support and work-focused healthcare compared to those receiving usual care in the cohort alone. We will recruit people aged 18-64, who have been out of work for at least 1 month, had pain for more than 3 months, and want to work. Initially, all (n = 228) will be recruited to an observational cohort study on the impact of being unemployed with persistent pain. We will then randomly select one in three to be offered the intervention. The primary outcome of sustained return to work will be measured using registry and self-reported data, while secondary outcomes include self-reported levels of health-related quality of life and physical and mental health. Outcomes will be measured at baseline and 3, 6, and 12 months post-randomisation. We will run a process evaluation parallel to the intervention exploring implementation, continuity of the intervention, reasons for participating, declining participation, and mechanisms behind cases of sustained return to work. An economic evaluation of the trial process will also be conducted.

DISCUSSION

The ReISE intervention is designed to increase work participation for people with persistent pain. The intervention has the potential to improve work ability by collaboratively navigating obstacles to working. If successful, the intervention may be a viable option for helping people in this population.

TRIAL REGISTRATION

ISRCTN Registry 85,437,524 Registered on 30 March 2022.

The effects of supported employment interventions in populations of people with conditions other than severe mental health: a systematic review

AIM

To assess the effectiveness of supported employment interventions for improving competitive employment in populations of people with conditions other than only severe mental illness.

BACKGROUND

Supported employment interventions have been extensively tested in severe mental illness populations. These approaches may be beneficial outside of these populations.

METHODS

We searched PubMed, Embase, CINAHL, PsycInfo, Web of Science, Scopus, JSTOR, PEDro, OTSeeker, and NIOSHTIC for trials including unemployed people with any condition and including severe mental illness if combined with other co-morbidities or other specific circumstances (e.g., homelessness). We excluded trials where inclusion was based on severe mental illness alone. Two reviewers independently assessed risk of bias (RoB v2.0) and four reviewers extracted data. We assessed rates of competitive employment as compared to traditional vocational rehabilitation or waiting list/services as usual.

FINDINGS

Ten randomised controlled trials (913 participants) were included. Supported employment was more effective than control interventions for improving competitive employment in seven trials: in people with affective disorders [risk ratio (RR) 10.61 (1.49, 75.38)]; mental disorders and justice involvement [RR 4.44 (1.36,14.46)]; veterans with posttraumatic stress disorder (PTSD) [RR 2.73 (1.64, 4.54)]; formerly incarcerated veterans [RR 2.17 (1.09, 4.33)]; people receiving methadone treatment [RR 11.5 (1.62, 81.8)]; veterans with spinal cord injury at 12 months [RR 2.46 (1.16, 5.22)] and at 24 months [RR 2.81 (1.98, 7.37)]; and young people not in employment, education, or training [RR 5.90 (1.91-18.19)]. Three trials did not show significant benefits from supported employment: populations of workers with musculoskeletal injuries [RR 1.38 (1.00, 1.89)]; substance abuse [RR 1.85 (0.65, 5.41)]; and formerly homeless people with mental illness [RR 1.55 (0.76, 3.15)]. Supported employment interventions may be beneficial to people from more diverse populations than those with severe mental illness alone. Defining competitive employment and increasing (and standardising) measurement of non-vocational outcomes may help to improve research in this area.

Core outcome set for preventive intervention trials in chronic and episodic migraine (COSMIG): an international, consensus-derived and multistakeholder initiative

OBJECTIVE

Typically, migraine prevention trials focus on reducing migraine days. This narrow focus may not capture all that is important to people with migraine. Inconsistency in outcome selection across trials limits the potential for data pooling and evidence synthesis. In response, we describe the development of core outcome set for migraine (COSMIG).

DESIGN

A two-stage approach sought to achieve international, multistakeholder consensus on both the core domain set and core measurement set. Following construction of a comprehensive list of outcomes, expert panellists (patients, healthcare professionals and researchers) completed a three-round electronic-Delphi study to support a reduction and prioritisation of core domains and outcomes. Participants in a consensus meeting finalised the core domains and methods of assessment. All stages were overseen by an international core team, including patient research partners.

RESULTS

There was a good representation of patients (episodic migraine (n=34) and chronic migraine (n=42)) and healthcare professionals (n=33) with high response and retention rates. The initial list of domains and outcomes was reduced from >50 to 7 core domains for consideration in the consensus meeting, during which a 2-domain core outcome set was agreed.

CONCLUSION

International and multistakeholder consensus emerged to describe a two-domain core outcome set for reporting research on preventive interventions for chronic and episodic migraine: migraine-specific pain and migraine-specific quality of life. Intensity of migraine pain assessed with an 11-point Numerical Rating Scale and the frequency as the number of headache/migraine days over a specified time period. Migraine-specific quality of life assessed using the Migraine Functional Impact Questionnaire.

Relative Performance of Machine Learning and Linear Regression in Predicting Quality of Life and Academic Performance of School Children in Norway: Data Analysis of a Quasi-Experimental Study

BACKGROUND

Machine learning techniques are increasingly being applied in health research. It is not clear how useful these approaches are for modeling continuous outcomes. Child quality of life is associated with parental socioeconomic status and physical activity and may be associated with aerobic fitness and strength. It is unclear whether diet or academic performance is associated with quality of life.

OBJECTIVE

The purpose of this study was to compare the predictive performance of machine learning techniques with that of linear regression in examining the extent to which continuous outcomes (physical activity, aerobic fitness, muscular strength, diet, and parental education) are predictive of academic performance and quality of life and whether academic performance and quality of life are associated.

METHODS

We modeled data from children attending 9 schools in a quasi-experimental study. We split data randomly into training and validation sets. Curvilinear, nonlinear, and heteroscedastic variables were simulated to examine the performance of machine learning techniques compared to that of linear models, with and without imputation.

RESULTS

We included data for 1711 children. Regression models explained 24% of academic performance variance in the real complete-case validation set, and up to 15% in quality of life. While machine learning techniques explained high proportions of variance in training sets, in validation, machine learning techniques explained approximately 0% of academic performance and 3% to 8% of quality of life. With imputation, machine learning techniques improved to 15% for academic performance. Machine learning outperformed regression for simulated nonlinear and heteroscedastic variables. The best predictors of academic performance in adjusted models were the child's mother having a master-level education (P<.001; β=1.98, 95% CI 0.25 to 3.71), increased television and computer use (P=.03; β=1.19, 95% CI 0.25 to 3.71), and dichotomized self-reported exercise (P=.001; β=2.47, 95% CI 1.08 to 3.87). For quality of life, self-reported exercise (P<.001; β=1.09, 95% CI 0.53 to 1.66) and increased television and computer use (P=.002; β=-0.95, 95% CI -1.55 to -0.36) were the best predictors. Adjusted academic performance was associated with quality of life (P=.02; β=0.12, 95% CI 0.02 to 0.22).

CONCLUSIONS

Linear regression was less prone to overfitting and outperformed commonly used machine learning techniques. Imputation improved the performance of machine learning, but not sufficiently to outperform regression. Machine learning techniques outperformed linear regression for modeling nonlinear and heteroscedastic relationships and may be of use in such cases. Regression with splines performed almost as well in nonlinear modeling. Lifestyle variables, including physical exercise, television and computer use, and parental education are predictive of academic performance or quality of life. Academic performance is associated with quality of life after adjusting for lifestyle variables and may offer another promising intervention target to improve quality of life in children.

Development and feasibility of an intervention featuring individual supported work placements to aid return to work for unemployed people living with chronic pain

BACKGROUND

Working in good jobs is associated with good health. High unemployment rates are reported in those disabled with musculoskeletal pain. Supported employment interventions work well for helping people with mental health difficulties to gain and retain employment. With adaptation, these may be useful for people with chronic pain. We aimed to develop and explore the feasibility of delivering such an adapted intervention.

METHODS

We developed an intervention and recruited unemployed people with chronic pain from NHS pain clinics and employment services. We trained case managers to assess participants and match them to six-week work placements in the Midlands and provide ongoing support to them and their managers. Participants attended a two-day work preparation session prior to placement. Outcome measures included quality of life at baseline, six- weeks, 14-weeks, and six-months, and return to work at 14-weeks and six-months. We held focus groups or interviews with stakeholders to examine acceptability and experiences of the intervention.

RESULTS

We developed an intervention consisting of work preparation sessions, work experience placements, and individualised employment support. We enrolled 31 people; 27 attended work preparation sessions, and 15 attended placements. Four of our participants started jobs during the study period. We are aware of two others starting jobs shortly after cessation of follow-up. We experienced challenges to recruitment in one area where we had many and diverse placement opportunities and good recruitment in another area where we had a smaller range of placement opportunities. All stakeholders found the intervention acceptable, and it was valued by those given a placement. While there was some disappointment among those not placed, this group still valued the work preparation sessions.

CONCLUSIONS

The developed intervention was acceptable to participants and partners. Trialling the developed intervention could be feasible with attention to three main processes. To ensure advanced availability of a sufficiently wide range of work placements in each area, multiple partners would be needed. Multiple recruitment sites and focus on employment services will yield better recruitment rates than reliance on NHS pain clinics. Maintaining an adequate follow-up response rate will likely require additional approaches with more than the usual effort.

Opportunities and challenges around adapting supported employment interventions for people with chronic low back pain: modified nominal group technique

PURPOSE

To identify and rank opportunities and challenges around adapting supported employment interventions for people with chronic low back pain (LBP).

METHODS

Delegates from an international back and neck research forum were invited to join an expert panel. A modified nominal group technique (NGT) was used with four stages: silent generation, round robin, clarification, and ranking. Ranked items were reported back and ratified by the panel.

RESULTS

Nine experienced researchers working in the fields related to LBP and disability joined the panel. Forty-eight items were generated and grouped into 12 categories of opportunities/challenges. Categories ranked most important related respectively to policy and legislation, ensuring operational integration across different systems, funding interventions, and managing attitudes towards work and health, workplace flexibility, availability of "good" work for this client group, dissonance between client and system aims, timing of interventions, and intervention development.

CONCLUSIONS

An expert panel believes the most important opportunities/challenges around adapting supporting employment interventions for people with chronic LBP are facilitating integration/communication between systems and institutions providing intervention components, optimising research outputs for informing policy needs, and encouraging discussion around funding mechanisms for research and interventions. Addressing these factors may help improve the quality and impact of future interventions.Implications for rehabilitationInteraction pathways between health, employment, and social systems need to be improved to effectively deliver intervention components that necessarily span these systems.Research-policy communication needs to be improved by researchers and policy makers, so that research outputs can be consumed by policy makers, and so that researchers recognise the gaps in knowledge needed to underpin policy.Improvements in research-policy communication and coordination would facilitate the delivery of research output at a time when it is likely to make the most impact on policy-making.Discussion and clarification surrounding funding mechanisms for research and interventions may facilitate innovation generally.

Obstacles to returning to work with chronic pain: in-depth interviews with people who are off work due to chronic pain and employers

Background

The global burden of chronic pain is growing with implications for both an ageing workforce and employers. Many obstacles are faced by people with chronic pain in finding employment and returning to work after a period of absence. Few studies have explored obstacles to return-to-work (RTW) from workers’ and employers’ perspectives. Here we explore views of both people in pain and employers about challenges to returning to work of people who are off work with chronic pain.

Methods

We did individual semi-structured interviews with people who were off work (unemployed or off sick) with chronic pain recruited from National Health Service (NHS) pain services and employment services, and employers from small, medium, and large public or private sector organisations. We analysed data using the Framework method.

Results

We interviewed 15 people off work with chronic pain and 10 employers. Obstacles to RTW for people with chronic pain spanned psychological, pain related, financial and economic, educational, and work-related domains. Employers were concerned about potential attitudinal obstacles, absence, ability of people with chronic pain to fulfil the job requirements, and the implications for workplace relationships. Views on disclosure of the pain condition were conflicting with more than half employers wanting early full disclosure and two-thirds of people with chronic pain declaring they would not disclose for fear of not getting a job or losing a job. Both employers and people with chronic pain thought that lack of confidence was an important obstacle. Changes to the job or work conditions (e.g. making reasonable adjustments, phased return, working from home or redeployment) were seen by both groups as facilitators. People with chronic pain wanted help in preparing to RTW, education for managers about pain and supportive working relationships.

Conclusions

People with chronic pain and employers may think differently in terms of perceptions of obstacles to RTW. Views appeared disparate in relation to disclosure of pain and when this needs to occur. They appeared to have more in common regarding opinions about how to facilitate successful RTW. Increased understanding of both perspectives may be used to inform the development of improved RTW interventions.

The work of return to work. Challenges of returning to work when you have chronic pain: a meta-ethnography

AIMS

To understand obstacles to returning to work, as perceived by people with chronic non-malignant pain and as perceived by employers, and to develop a conceptual model.

DESIGN

Synthesis of qualitative research using meta-ethnography.

DATA SOURCES

Eleven bibliographic databases from inception to April 2017 supplemented by citation tracking.

REVIEW METHODS

We used the methods of meta-ethnography. We identified concepts and conceptual categories, and developed a conceptual model and line of argument.

RESULTS

We included 41 studies. We identified three core categories in the conceptual model: managing pain, managing work relationships and making workplace adjustments. All were influenced by societal expectations in relation to work, self (self-belief, self-efficacy, legitimacy, autonomy and the meaning of work for the individual), health/illness/pain representations, prereturn to work support and rehabilitation, and system factors (healthcare, workplace and social security). A mismatch of expectations between the individual with pain and the workplace contributed to a feeling of being judged and difficulties asking for help. The ability to navigate obstacles and negotiate change underpinned mastering return to work despite the pain. Where this ability was not apparent, there could be a downward spiral resulting in not working.

CONCLUSIONS

For people with chronic pain, and for their employers, navigating obstacles to return to work entails balancing the needs of (1) the person with chronic pain, (2) work colleagues and (3) the employing organisation. Managing pain, managing work relationships and making workplace adjustments appear to be central, but not straightforward, and require substantial effort to culminate in a successful return to work.

Research ethics oversight in Norway: structure, function, and challenges

BACKGROUND

While the development and evaluation of clinical ethics services in Norway has been recognized internationally, the country's research ethics infrastructure at times may have been less well developed. In 2016, media interest in the controversial nature of some health services research and pilot studies highlighted gaps in the system with certain types of research having no clear mechanisms through which they may be given due independent consideration. It is not clear that new legislation, implemented in 2017, will address this problem. We explore relevant law, committee scope, and the function of the system. We show that 1) Norwegian law provides for ethics assessment for all forms of health research; 2) regional RECs in Norway might not have always enforced this provision, considering some interventional health services research to be outside their remit; and 3) Norwegian law does not explicity provide for local/university RECs, meaning that, in practice, there may be no readily accessible mechanisms for the assessment of research that is excluded by regional RECs. This may include health services research, pilot studies, and undergraduate research. New 2017 legislation has no effect on this specifically but focuses on institutions regulating researcher activity. This may place researchers in the difficult situation of on one hand, needing to hold to recognized ethical standards, while on the other, not readily having access to independent committee scrutiny to facilitate consistent operation with these standards.

CONCLUSION

To support researchers in Norway and to protect the public, it may be necessary either to widen the regional RECs' remit or to make legislative alterations that permit and do not discourage the existence of local RECs.

Responsiveness, Reliability, and Minimally Important and Minimal Detectable Changes of 3 Electronic Patient-Reported Outcome Measures for Low Back Pain: Validation Study

Background

The Roland Morris Disability Questionnaire (RMDQ), visual analog scale (VAS) of pain intensity, and numerical rating scale (NRS) are among the most commonly used outcome measures in trials of interventions for low back pain. Their use in paper form is well established. Few data are available on the metric properties of electronic counterparts.

Objective

The goal of our research was to establish responsiveness, minimally important change (MIC) thresholds, reliability, and minimal detectable change at a 95% level (MDC₉₅) for electronic versions of the RMDQ, VAS, and NRS as delivered via iOS and Android apps and Web browser.

Methods

We recruited adults with low back pain who visited osteopaths. We invited participants to complete the eRMDQ, eVAS, and eNRS at baseline, 1 week, and 6 weeks along with a health transition question at 1 and 6 weeks. Data from participants reporting recovery were used in MIC and responsiveness analyses using receiver operator characteristic (ROC) curves and areas under the ROC curves (AUCs). Data from participants reporting stability were used for analyses of reliability (intraclass correlation coefficient [ICC] agreement) and MDC₉₅.

Results

We included 442 participants. At 1 and 6 weeks, ROC AUCs were 0.69 (95% CI 0.59 to 0.80) and 0.67 (95% CI 0.46 to 0.87) for the eRMDQ, 0.69 (95% CI 0.58 to 0.80) and 0.74 (95% CI 0.53 to 0.95) for the eVAS, and 0.73 (95% CI 0.66 to 0.80) and 0.81 (95% CI 0.69 to 0.92) for the eNRS, respectively. Associated MIC thresholds were estimated as 1 (0 to 2) and 2 (–1 to 5), 13 (9 to 17) and 7 (–12 to 26), and 2 (1 to 3) and 1 (0 to 2) points, respectively. Over a 1-week period in participants categorized as “stable” and “about the same” using the transition question, ICCs were 0.87 (95% CI 0.66 to 0.95) and 0.84 (95% CI 0.73 to 0.91) for the eRMDQ with MDC₉₅ of 4 and 5, 0.31 (95% CI –0.25 to 0.71) and 0.61 (95% CI 0.36 to 0.77) for the eVAS with MDC₉₅ of 39 and 34, and 0.52 (95% CI 0.14 to 0.77) to 0.67 (95% CI 0.51 to 0.78) with MDC₉₅ of 4 and 3 for the eNRS.

Conclusions

The eRMDQ was reliable with borderline adequate responsiveness. The eNRS was responsive with borderline reliability. While the eVAS had adequate responsiveness, it did not have an attractive reliability profile. Thus, the eNRS might be preferred over the eVAS for measuring pain intensity. The observed electronic outcome measures’ metric properties are within the ranges of values reported in the literature for their paper counterparts and are adequate for measuring changes in a low back pain population.

Consensus generation of a minimum set of outcome measures for auditing glaucoma surgery outcomes-a Delphi exercise

PURPOSE

To identify the key set of glaucoma surgery outcome measures considered most important and practical to collect by glaucoma specialists.

METHODS

One hundred two glaucoma specialists (57 members of the UK and Eire Glaucoma Society (UKEGS) and 45 members of the European Glaucoma society (EGS)) took part in an Online Delphi exercise. The RAND/UCLA appropriateness method was used analyse data from each round and generate a disagreement index.

RESULTS

Participants agreed on 13 baseline data points and 12 outcomes that were considered important and practical to collect. For intraocular pressure (IOP) percentage reduction in IOP from baseline (last three IOP readings pre-op) and reduction below a specified target were considered important. For visual fields, change in a global visual field index, e.g. MD, and development of progression as assessed by linear regression were considered important. From a safety perspective, any visual loss resulting in a doubling of the minimal angle of resolution, loss of 5 dB or more of visual field or development of advanced field loss (Hodapp Parrish Anderson Stage 4) was considered important. The importance of routinely using patient reported outcome measures (PROMs) was highlighted. Consensus suggested that outcomes of glaucoma treatments should be reported at 1, 5 and 10 years.

CONCLUSIONS

There was broad consensus on a minimum dataset for reporting the outcomes of glaucoma surgery and outcome measurement intervals.

Inclusion and exclusion criteria used in non-specific low back pain trials: a review of randomised controlled trials published between 2006 and 2012

Background

Low back pain is a common health complaint resulting in substantial economic burden. Each year, upwards of 20 randomised controlled trials (RCTs) evaluating interventions for non-specific low back pain are published. Use of the term non-specific low back pain has been criticised on the grounds of encouraging heterogeneity and hampering interpretation of findings due to possible heterogeneous causes, challenging meta-analyses. We explored selection criteria used in trials of treatments for nsLBP.

Methods

A systematic review of English-language reports of RCTs in nsLBP population samples, published between 2006 and 2012, identified from MEDLINE, EMBASE, and the Cochrane Library databases, using a mixed-methods approach to analysis. Study inclusion and exclusion criteria were extracted, thematically categorised, and then descriptive statistics were used to summarise the prevalence by emerging category.

Results

We included 168 studies. Two inclusion themes (anatomical area, and symptoms and signs) were identified. Anatomical area was most reported as between costal margins and gluteal folds (n = 8, 5%), while low back pain (n = 150, 89%) with or without referred leg pain (n = 27, 16%) was the most reported symptom. Exclusion criteria comprised 21 themes. Previous or scheduled surgery (n = 84, 50%), pregnancy (n = 81, 48%), malignancy (n = 78, 46%), trauma (n = 63, 37%) and psychological conditions (n = 58, 34%) were the most common. Sub-themes of exclusion criteria mostly related to neurological signs and symptoms: nerve root compromise (n = 44, 26%), neurological signs (n = 34, 20%) or disc herniation (n = 30, 18%). Specific conditions that were most often exclusion criteria were spondylolisthesis (n = 35, 21%), spinal stenosis (n = 31, 18%) or osteoporosis (n = 27, 16%).

Conclusion

RCTs of interventions for non-specific low back pain have incorporated diverse inclusion and exclusion criteria. Guidance on standardisation of inclusion and exclusion criteria for nsLBP trials will increase clinical homogeneity, facilitating greater interpretation of between-trial comparisons and meta-analyses. We propose a template for reporting inclusion and exclusion criteria.

Electronic supplementary material

The online version of this article (10.1186/s12891-018-2034-6) contains supplementary material, which is available to authorized users.

A Systematic Review of Outcome Measures Use, Analytical Approaches, Reporting Methods, and Publication Volume by Year in Low Back Pain Trials Published between 1980 and 2012: Respice, adspice, et prospice

BACKGROUND

Increasing patient-reported outcome measures in the 1980s and 1990s led to the development of recommendations at the turn of the millennium for standardising outcome measures in non-specific low back pain (LBP) trials. Whether these recommendations impacted use is unclear. Previous work has examined citation counts, but actual use and change over time, has not been explored. Since 2011, there has been some consensus on the optimal methods for reporting back pain trial outcomes. We explored reporting practice, outcome measure use, and publications over time.

METHODS

We performed a systematic review of LBP trials, searching the European Guidelines for the management of LBP, extending the search to 2012. We abstracted data on publications by year, outcome measure use, analytical approach, and approaches taken to reporting trials outcomes. Data were analysed using descriptive statistics and regression analyses.

RESULTS

We included 401 trials. The number of published trials per year has increased by a factor of 4.5 from 5.4 (1980-1999) to 24.4 (2000-2012). The most commonly used outcome measures have been the Visual Analogue Scale for pain intensity, which has slowly increased in use since 1980/81 from 20% to 60% of trials by 2012, and the Roland-Morris Disability Questionnaire, which rose to 55% in 2002/2003, and then fell back to 28% by 2012. Most trialists (85%) report between-group mean differences. Few (8%) report individual improvements, and some (4%) report only within-group analyses. Student's t test, ANOVA, and ANCOVA regression, or mixed models, were the most common approaches to analysis.

CONCLUSIONS

Recommendations for standardising outcomes may have had a limited or inconsistent effect on practice. Since the research community is again considering outcome measures and modifying recommendations, groups offering recommendations should be cognisant that better ways of generating trialist buy-in may be required in order for their recommendations to have impact.

The effect of journal impact factor, reporting conflicts, and reporting funding sources, on standardized effect sizes in back pain trials: a systematic review and meta-regression

BACKGROUND

Low back pain is a common and costly health complaint for which there are several moderately effective treatments. In some fields there is evidence that funder and financial conflicts are associated with trial outcomes. It is not clear whether effect sizes in back pain trials relate to journal impact factor, reporting conflicts of interest, or reporting funding.

METHODS

We performed a systematic review of English-language papers reporting randomised controlled trials of treatments for non-specific low back pain, published between 2006-2012. We modelled the relationship using 5-year journal impact factor, and categories of reported of conflicts of interest, and categories of reported funding (reported none and reported some, compared to not reporting these) using meta-regression, adjusting for sample size, and publication year. We also considered whether impact factor could be predicted by the direction of outcome, or trial sample size.

RESULTS

We could abstract data to calculate effect size in 99 of 146 trials that met our inclusion criteria. Effect size is not associated with impact factor, reporting of funding source, or reporting of conflicts of interest. However, explicitly reporting 'no trial funding' is strongly associated with larger absolute values of effect size (adjusted β=1.02 (95 % CI 0.44 to 1.59), P=0.001). Impact factor increases by 0.008 (0.004 to 0.012) per unit increase in trial sample size (P<0.001), but does not differ by reported direction of the LBP trial outcome (P=0.270).

CONCLUSIONS

The absence of associations between effect size and impact factor, reporting sources of funding, and conflicts of interest reflects positively on research and publisher conduct in the field. Strong evidence of a large association between absolute magnitude of effect size and explicit reporting of 'no funding' suggests authors of unfunded trials are likely to report larger effect sizes, notwithstanding direction. This could relate in part to quality, resources, and/or how pragmatic a trial is.

The MAPS Reporting Statement for Studies Mapping onto Generic Preference-Based Outcome Measures: Explanation and Elaboration

BACKGROUND

The process of "mapping" is increasingly being used to predict health utilities, for application within health economic evaluations, using data on other indicators or measures of health. Guidance for the reporting of mapping studies is currently lacking.

OBJECTIVE

The overall objective of this research was to develop a checklist of essential items, which authors should consider when reporting mapping studies. The MAPS (MApping onto Preference-based measures reporting Standards) statement is a checklist, which aims to promote complete and transparent reporting by researchers. This paper provides a detailed explanation and elaboration of the items contained within the MAPS statement.

METHODS

In the absence of previously published reporting checklists or reporting guidance documents, a de novo list of reporting items and accompanying explanations was created. A two-round, modified Delphi survey, with representatives from academia, consultancy, health technology assessment agencies and the biomedical journal editorial community, was used to identify a list of essential reporting items from this larger list.

RESULTS

From the initial de novo list of 29 candidate items, a set of 23 essential reporting items was developed. The items are presented numerically and categorised within six sections, namely, (i) title and abstract, (ii) introduction, (iii) methods, (iv) results, (v) discussion and (vi) other. For each item, we summarise the recommendation, illustrate it using an exemplar of good reporting practice identified from the published literature, and provide a detailed explanation to accompany the recommendation.

CONCLUSIONS

It is anticipated that the MAPS statement will promote clarity, transparency and completeness of reporting of mapping studies. It is targeted at researchers developing mapping algorithms, peer reviewers and editors involved in the manuscript review process for mapping studies, and the funders of the research. The MAPS working group plans to assess the need for an update of the reporting checklist in 5 years' time.

Preferred Reporting Items for Studies Mapping onto Preference-Based Outcome Measures: The MAPS Statement

'Mapping' onto generic preference-based outcome measures is increasingly being used as a means of generating health utilities for use within health economic evaluations. Despite the publication of technical guides for the conduct of mapping research, guidance for the reporting of mapping studies is currently lacking. The MAPS (MApping onto Preference-based measures reporting Standards) statement is a new checklist, which aims to promote complete and transparent reporting of mapping studies. The primary audiences for the MAPS statement are researchers reporting mapping studies, the funders of the research, and peer reviewers and editors involved in assessing mapping studies for publication. A de novo list of 29 candidate reporting items and accompanying explanations was created by a working group comprising six health economists and one Delphi methodologist. Following a two-round modified Delphi survey with representatives from academia, consultancy, health technology assessment agencies and the biomedical journal editorial community, a final set of 23 items deemed essential for transparent reporting, and accompanying explanations, was developed. The items are contained in a user-friendly 23-item checklist. They are presented numerically and categorised within six sections, namely: (1) title and abstract; (2) introduction; (3) methods; (4) results; (5) discussion; and (6) other. The MAPS statement is best applied in conjunction with the accompanying MAPS explanation and elaboration document. It is anticipated that the MAPS statement will improve the clarity, transparency and completeness of reporting of mapping studies. To facilitate dissemination and uptake, the MAPS statement is being co-published by seven health economics and quality-of-life journals, and broader endorsement is encouraged. The MAPS working group plans to assess the need for an update of the reporting checklist in 5 years' time.

The MAPS Reporting Statement for Studies Mapping onto Generic Preference-Based Outcome Measures: Explanation and Elaboration

BACKGROUND

The process of "mapping" is increasingly being used to predict health utilities, for application within health economic evaluations, using data on other indicators or measures of health. Guidance for the reporting of mapping studies is currently lacking.

OBJECTIVE

The overall objective of this research was to develop a checklist of essential items, which authors should consider when reporting mapping studies. The MAPS (MApping onto Preference-based measures reporting Standards) statement is a checklist, which aims to promote complete and transparent reporting by researchers. This paper provides a detailed explanation and elaboration of the items contained within the MAPS statement.

METHODS

In the absence of previously published reporting checklists or reporting guidance documents, a de novo list of reporting items and accompanying explanations was created. A two-round, modified Delphi survey, with representatives from academia, consultancy, health technology assessment agencies and the biomedical journal editorial community, was used to identify a list of essential reporting items from this larger list.

RESULTS

From the initial de novo list of 29 candidate items, a set of 23 essential reporting items was developed. The items are presented numerically and categorised within six sections, namely, (i) title and abstract, (ii) introduction, (iii) methods, (iv) results, (v) discussion and (vi) other. For each item, we summarise the recommendation, illustrate it using an exemplar of good reporting practice identified from the published literature, and provide a detailed explanation to accompany the recommendation.

CONCLUSIONS

It is anticipated that the MAPS statement will promote clarity, transparency and completeness of reporting of mapping studies. It is targeted at researchers developing mapping algorithms, peer reviewers and editors involved in the manuscript review process for mapping studies, and the funders of the research. The MAPS working group plans to assess the need for an update of the reporting checklist in 5 years' time.

Preferred Reporting Items for Studies Mapping onto Preference-Based Outcome Measures: The MAPS Statement

BACKGROUND

'Mapping' onto generic preference-based outcome measures is increasingly being used as a means of generating health utilities for use within health economic evaluations. Despite publication of technical guides for the conduct of mapping research, guidance for the reporting of mapping studies is currently lacking. The MAPS (MApping onto Preference-based measures reporting Standards) statement is a new checklist, which aims to promote complete and transparent reporting of mapping studies.

METHODS

In the absence of previously published reporting checklists or reporting guidance documents, a de novo list of reporting items was created by a working group comprising six health economists and one Delphi methodologist. A two-round, modified Delphi survey, with representatives from academia, consultancy, health technology assessment agencies and the biomedical journal editorial community, was used to identify a list of essential reporting items from this larger list.

RESULTS

From the initial de novo list of 29 candidate items, a set of 23 essential reporting items was developed. The items are presented numerically and categorized within six sections: (1) title and abstract; (2) introduction; (3) methods; (4) results; (5) discussion; and (6) other. The MAPS statement is best applied in conjunction with the accompanying MAPS Explanation and Elaboration paper.

CONCLUSION

It is anticipated that the MAPS statement will improve the clarity, transparency and completeness of the reporting of mapping studies. To facilitate dissemination and uptake, the MAPS statement is being co-published by seven health economics and quality-of-life journals, and broader endorsement is encouraged. The MAPS working group plans to assess the need for an update of the reporting checklist in 5 years' time.

Preferred reporting items for studies mapping onto preference-based outcome measures: the MAPS statement

BACKGROUND AND AIM

'Mapping' onto generic preference-based outcome measures is increasingly being used as a means of generating health utilities for use within health economic evaluations. Despite publication of technical guides for the conduct of mapping research, guidance for the reporting of mapping studies is currently lacking. The MApping onto Preference-based measures reporting Standards (MAPS) statement is a new checklist, which aims to promote complete and transparent reporting of mapping studies.

METHODS

In the absence of previously published reporting checklists or reporting guidance documents, a de novo list of reporting items was created by a working group comprised of six health economists and one Delphi methodologist. A two-round, modified Delphi survey with representatives from academia, consultancy, health technology assessment agencies and the biomedical journal editorial community was used to identify a list of essential reporting items from this larger list.

RESULTS

From the initial de novo list of 29 candidate items, a set of 23 essential reporting items was developed. The items are presented numerically and categorised within six sections, namely (1) title and abstract; (2) introduction; (3) methods; (4) results; (5) discussion; and (6) other. The MAPS statement is best applied in conjunction with the accompanying MAPS explanation and elaboration document.

CONCLUSIONS

It is anticipated that the MAPS statement will improve the clarity, transparency and completeness of reporting of mapping studies. To facilitate dissemination and uptake, the MAPS statement is being co-published by seven health economics and quality of life journals, and broader endorsement is encouraged. The MAPS working group plans to assess the need for an update of the reporting checklist in 5 years' time.

Preferred Reporting Items for Studies Mapping onto Preference-Based Outcome Measures: The MAPS Statement

BACKGROUND

"Mapping" onto generic preference-based outcome measures is increasingly being used as a means of generating health utilities for use within health economic evaluations. Despite the publication of technical guides for the conduct of mapping research, guidance for the reporting of mapping studies is currently lacking. The MAPS (MApping onto Preference-based measures reporting Standards) statement is a new checklist that aims to promote complete and transparent reporting of mapping studies.

METHODS

In the absence of previously published reporting checklists or reporting guidance documents, a de novo list of reporting items was created by a working group comprised of 6 health economists and 1 Delphi methodologist. A 2-round, modified Delphi survey with representatives from academia, consultancy, health technology assessment agencies, and the biomedical journal editorial community was used to identify a list of essential reporting items from this larger list.

RESULTS

From the initial de novo list of 29 candidate items, a set of 23 essential reporting items was developed. The items are presented numerically and categorized within 6 sections, namely: (i) title and abstract; (ii) introduction; (iii) methods; (iv) results; (v) discussion; and (vi) other. The MAPS statement is best applied in conjunction with the accompanying MAPS explanation and elaboration document.

CONCLUSIONS

It is anticipated that the MAPS statement will improve the clarity, transparency, and completeness of reporting of mapping studies. To facilitate dissemination and uptake, the MAPS statement is being co-published by 7 health economics and quality-of-life journals, and broader endorsement is encouraged. The MAPS working group plans to assess the need for an update of the reporting checklist in 5 years.

Preferred reporting items for studies mapping onto preference-based outcome measures: The MAPS statement

‘Mapping’ onto generic preference-based outcome measures is increasingly being used as a means of generating health utilities for use within health economic evaluations. Despite publication of technical guides for the conduct of mapping research, guidance for the reporting of mapping studies is currently lacking. The MAPS (MApping onto Preference-based measures reporting Standards) statement is a new checklist, which aims to promote complete and transparent reporting of mapping studies. The primary audiences for the MAPS statement are researchers reporting mapping studies, the funders of the research, and peer reviewers and editors involved in assessing mapping studies for publication.

A de novo list of 29 candidate reporting items and accompanying explanations was created by a working group comprised of six health economists and one Delphi methodologist. Following a two-round, modified Delphi survey with representatives from academia, consultancy, health technology assessment agencies and the biomedical journal editorial community, a final set of 23 items deemed essential for transparent reporting, and accompanying explanations, was developed. The items are contained in a user friendly 23 item checklist. They are presented numerically and categorised within six sections, namely: (i) title and abstract; (ii) introduction; (iii) methods; (iv) results; (v) discussion; and (vi) other. The MAPS statement is best applied in conjunction with the accompanying MAPS explanation and elaboration document.

It is anticipated that the MAPS statement will improve the clarity, transparency and completeness of reporting of mapping studies. To facilitate dissemination and uptake, the MAPS statement is being co-published by eight health economics and quality of life journals, and broader endorsement is encouraged. The MAPS working group plans to assess the need for an update of the reporting checklist in five years’ time.

This statement was published jointly in Applied Health Economics and Health Policy, Health and Quality of Life Outcomes, International Journal of Technology Assessment in Health Care, Journal of Medical Economics, Medical Decision Making, PharmacoEconomics, and Quality of Life Research.

Primary outcome measure use in back pain trials may need radical reassessment

Background

The answers to patient reported outcome measures and global transition questions for back pain can be discordant. For example, the most commonly used outcome measure in back pain trials, the Roland Morris Disability Questionnaire (RMDQ), can show improvement even though participants say that their back pain is worse. This gives cause for concern as transition questions are used as anchors to estimate minimally important change (MIC) thresholds on patient reported outcome measures such as the RMDQ. We aimed to explore and compare what people with back pain think when they respond to a transition question and when they complete the RMDQ.

Methods

We purposively sampled people enrolled on a back pain randomised controlled trial who completed the RMDQ and two transition questions. One enquired about change in ability to perform tasks, the other about change in back pain. We sampled participants with discordance (in both directions), and participants with concordant scores. We explored participants’ thought processes using in-depth interviews.

Results

We completed 35 in-depth interviews. People with discordant RMDQ change and transition question responses attend to different factors when responding to transition questions compared to people with concordant scores. In particular, those for whom the RMDQ change indicated greater improvement than transition questions, prioritised their pain ahead of functional disability. When completing the RMDQ, participants’ thought processes were comparatively more objective, and specific to each statement.

Conclusion

Approaches to primary outcome assessment in back pain needs re-assessment. The RMDQ may be unsuitable for use as a primary outcome measure since patients may not attend to thinking about their back pain when completing it: patients’ abilities to cope with tasks can be independent of the change in their back pain. Some participants who improve on the RMDQ consider themselves globally worse. As transition questions can be driven by pain and other physical factors, transition questions should not be used to anchor minimally important change thresholds on the RMDQ.

Electronic supplementary material

The online version of this article (doi:10.1186/s12891-015-0534-1) contains supplementary material, which is available to authorized users.

Preferred reporting items for studies mapping onto preference-based outcome measures: The MAPS statement

'Mapping' onto generic preference-based outcome measures is increasingly being used as a means of generating health utilities for use within health economic evaluations. Despite publication of technical guides for the conduct of mapping research, guidance for the reporting of mapping studies is currently lacking. The MAPS (MApping onto Preference-based measures reporting Standards) statement is a new checklist, which aims to promote complete and transparent reporting of mapping studies. The primary audiences for the MAPS statement are researchers reporting mapping studies, the funders of the research, and peer reviewers and editors involved in assessing mapping studies for publication. A de novo list of 29 candidate reporting items and accompanying explanations was created by a working group comprised of six health economists and one Delphi methodologist. Following a two-round, modified Delphi survey with representatives from academia, consultancy, health technology assessment agencies and the biomedical journal editorial community, a final set of 23 items deemed essential for transparent reporting, and accompanying explanations, was developed. The items are contained in a user friendly 23 item checklist. They are presented numerically and categorised within six sections, namely: (i) title and abstract; (ii) introduction; (iii) methods; (iv) results; (v) discussion; and (vi) other. The MAPS statement is best applied in conjunction with the accompanying MAPS explanation and elaboration document. It is anticipated that the MAPS statement will improve the clarity, transparency and completeness of reporting of mapping studies. To facilitate dissemination and uptake, the MAPS statement is being co-published by seven health economics and quality of life journals, and broader endorsement is encouraged. The MAPS working group plans to assess the need for an update of the reporting checklist in five years' time.

A systematic review and meta-synthesis of the impact of low back pain on people's lives

Background

Low back pain (LBP) is a common and costly problem that many interpret within a biopsychosocial model. There is renewed concern that core-sets of outcome measures do not capture what is important. To inform debate about the coverage of back pain outcome measure core-sets, and to suggest areas worthy of exploration within healthcare consultations, we have synthesised the qualitative literature on the impact of low back pain on people’s lives.

Methods

Two reviewers searched CINAHL, Embase, PsycINFO, PEDro, and Medline, identifying qualitative studies of people’s experiences of non-specific LBP. Abstracted data were thematic coded and synthesised using a meta-ethnographic, and a meta-narrative approach.

Results

We included 49 papers describing 42 studies. Patients are concerned with engagement in meaningful activities; but they also want to be believed and have their experiences and identity, as someone ‘doing battle’ with pain, validated. Patients seek diagnosis, treatment, and cure, but also reassurance of the absence of pathology. Some struggle to meet social expectations and obligations. When these are achieved, the credibility of their pain/disability claims can be jeopardised. Others withdraw, fearful of disapproval, or unable or unwilling to accommodate social demands. Patients generally seek to regain their pre-pain levels of health, and physical and emotional stability. After time, this can be perceived to become unrealistic and some adjust their expectations accordingly.

Conclusions

The social component of the biopsychosocial model is not well represented in current core-sets of outcome measures. Clinicians should appreciate that the broader impact of low back pain includes social factors; this may be crucial to improving patients’ experiences of health care. Researchers should consider social factors to help develop a portfolio of more relevant outcome measures.

A systematic review of cluster randomised trials in residential facilities for older people suggests how to improve quality

BACKGROUND

Previous reviews of cluster randomised trials have been critical of the quality of the trials reviewed, but none has explored determinants of the quality of these trials in a specific field over an extended period of time. Recent work suggests that correct conduct and reporting of these trials may require more than published guidelines. In this review, our aim was to assess the quality of cluster randomised trials conducted in residential facilities for older people, and to determine whether (1) statistician involvement in the trial and (2) strength of journal endorsement of the Consolidated Standards of Reporting Trials (CONSORT) statement influence quality.

METHODS

We systematically identified trials randomising residential facilities for older people, or parts thereof, without language restrictions, up to the end of 2010, using National Library of Medicine (Medline) via PubMed and hand-searching. We based quality assessment criteria largely on the extended CONSORT statement for cluster randomised trials. We assessed statistician involvement based on statistician co-authorship, and strength of journal endorsement of the CONSORT statement from journal websites.

RESULTS

73 trials met our inclusion criteria. Of these, 20 (27%) reported accounting for clustering in sample size calculations and 54 (74%) in the analyses. In 29 trials (40%), methods used to identify/recruit participants were judged by us to have potentially caused bias or reporting was unclear to reach a conclusion. Some elements of quality improved over time but this appeared not to be related to the publication of the extended CONSORT statement for these trials. Trials with statistician/epidemiologist co-authors were more likely to account for clustering in sample size calculations (unadjusted odds ratio 5.4, 95% confidence interval 1.1 to 26.0) and analyses (unadjusted OR 3.2, 1.2 to 8.5). Journal endorsement of the CONSORT statement was not associated with trial quality.

CONCLUSIONS

Despite international attempts to improve methods in cluster randomised trials, important quality limitations remain amongst these trials in residential facilities. Statistician involvement on trial teams may be more effective in promoting quality than further journal endorsement of the CONSORT statement. Funding bodies and journals should promote statistician involvement and co-authorship in addition to adherence to CONSORT guidelines.

Quality of cluster randomized controlled trials in oral health: a systematic review of reports published between 2005 and 2009

OBJECTIVES

To assess the quality of methods and reporting of recently published cluster randomized trials (CRTs) in oral health.

METHODS

We searched PubMed for CRTs that included at least one oral health-related outcome and were published from 2005 to 2009 inclusive. We developed a list of criteria for assessing trial quality and reporting. This was influenced largely by the extended CONSORT statement for CRTs but also included criteria suggested by other authors. We examined the extent to which trials were consistent with these criteria.

RESULTS

Twenty-three trials were included in the review. In 15 (65%) trials, clustering had been accounted for in sample size calculations, and in 18 (78%) authors had accounted for clustering in analysis. Intraclass correlation coefficients (ICCs) were reported for eight (35%) trials; the outcome assessor was reported as having been blinded to allocation in 12 (52%) trials; 17 (74%) described eligibility criteria at individual level, but only nine (39%) described such criteria at cluster level. Sixteen of 20 trials (80%), in which individuals were recruited, reported that individual informed consent was obtained.

CONCLUSIONS

These results suggest that the quality of recent CRTs in oral health is relatively high and appears to compare favourably with other fields. However, there remains room for improvement. Authors of future trials should endeavour to ensure sample size calculations and analyses properly account for clustering (and are reported as such), consider the potential for recruitment/identification bias at the design stage, describe the steps taken to avoid this in the final report and report observed ICCs and cluster-level eligibility criteria.

Reporting outcomes of back pain trials: a modified Delphi study

BACKGROUND

Low back pain is a common and expensive health complaint. Many low back pain trials have been conducted, but these are reported in a variety of ways and are often difficult to interpret.

AIM

To facilitate consensus on a statement recommending reporting methods for future low back pain trials.

METHODS

We presented experts with clinicians' views on different reporting methods and asked them to rate and comment on the suitability reporting methods for inclusion in a standardized set. Panellists developed a statement of recommendation over three online rounds. We used a modified Delphi process and the RAND/UCLA appropriateness method as a formal framework for establishing appropriateness and quantifying panel disagreement.

RESULTS

A group of 63 experts from 14 countries participated. Consensus was reached on a statement recommending that the continuous patient-reported outcomes commonly used in back pain trials, are reported using between-group mean differences (accompanied by minimally important difference (between-group/population-level) thresholds where these exist), the proportion of participants improving and deteriorating according to established and relevant minimally important change thresholds, and the number needed to treat; all with 95% confidence intervals. Outcomes may additionally be reported using alternative approaches (e.g. relative risks, odds ratios, or standardized mean difference) according to the needs of a particular trial.

CONCLUSIONS

A group of back pain experts reached a high level of consensus on a statement recommending reporting methods for patient-reported outcomes in future low back pain trials. The statement has the potential to increase interpretability and improve patient care.

Methodological criteria for the assessment of moderators in systematic reviews of randomised controlled trials: a consensus study

BACKGROUND

Current methodological guidelines provide advice about the assessment of sub-group analysis within RCTs, but do not specify explicit criteria for assessment. Our objective was to provide researchers with a set of criteria that will facilitate the grading of evidence for moderators, in systematic reviews.

METHOD

We developed a set of criteria from methodological manuscripts (n = 18) using snowballing technique, and electronic database searches. Criteria were reviewed by an international Delphi panel (n = 21), comprising authors who have published methodological papers in this area, and researchers who have been active in the study of sub-group analysis in RCTs. We used the Research ANd Development/University of California Los Angeles appropriateness method to assess consensus on the quantitative data. Free responses were coded for consensus and disagreement. In a subsequent round additional criteria were extracted from the Cochrane Reviewers' Handbook, and the process was repeated.

RESULTS

The recommendations are that meta-analysts report both confirmatory and exploratory findings for sub-groups analysis. Confirmatory findings must only come from studies in which a specific theory/evidence based a-priori statement is made. Exploratory findings may be used to inform future/subsequent trials. However, for inclusion in the meta-analysis of moderators, the following additional criteria should be applied to each study: Baseline factors should be measured prior to randomisation, measurement of baseline factors should be of adequate reliability and validity, and a specific test of the interaction between baseline factors and interventions must be presented.

CONCLUSIONS

There is consensus from a group of 21 international experts that methodological criteria to assess moderators within systematic reviews of RCTs is both timely and necessary. The consensus from the experts resulted in five criteria divided into two groups when synthesising evidence: confirmatory findings to support hypotheses about moderators and exploratory findings to inform future research. These recommendations are discussed in reference to previous recommendations for evaluating and reporting moderator studies.

Estimating the number needed to treat from continuous outcomes in randomised controlled trials: methodological challenges and worked example using data from the UK Back Pain Exercise and Manipulation (BEAM) trial

Background

Reporting numbers needed to treat (NNT) improves interpretability of trial results. It is unusual that continuous outcomes are converted to numbers of individual responders to treatment (i.e., those who reach a particular threshold of change); and deteriorations prevented are only rarely considered. We consider how numbers needed to treat can be derived from continuous outcomes; illustrated with a worked example showing the methods and challenges.

Methods

We used data from the UK BEAM trial (n = 1, 334) of physical treatments for back pain; originally reported as showing, at best, small to moderate benefits. Participants were randomised to receive 'best care' in general practice, the comparator treatment, or one of three manual and/or exercise treatments: 'best care' plus manipulation, exercise, or manipulation followed by exercise. We used established consensus thresholds for improvement in Roland-Morris disability questionnaire scores at three and twelve months to derive NNTs for improvements and for benefits (improvements gained+deteriorations prevented).

Results

At three months, NNT estimates ranged from 5.1 (95% CI 3.4 to 10.7) to 9.0 (5.0 to 45.5) for exercise, 5.0 (3.4 to 9.8) to 5.4 (3.8 to 9.9) for manipulation, and 3.3 (2.5 to 4.9) to 4.8 (3.5 to 7.8) for manipulation followed by exercise. Corresponding between-group mean differences in the Roland-Morris disability questionnaire were 1.6 (0.8 to 2.3), 1.4 (0.6 to 2.1), and 1.9 (1.2 to 2.6) points.

Conclusion

In contrast to small mean differences originally reported, NNTs were small and could be attractive to clinicians, patients, and purchasers. NNTs can aid the interpretation of results of trials using continuous outcomes. Where possible, these should be reported alongside mean differences. Challenges remain in calculating NNTs for some continuous outcomes.

Trial Registration

UK BEAM trial registration: ISRCTN32683578.