Identification and prioritisation of items for a draft participant-reported questionnaire to measure use of social care, informal care, aids and adaptations

BACKGROUND

Resource-use measurement is integral for assessing cost-effectiveness within trial-based economic evaluations. Methods for gathering resource-use data from participants are not well developed, with questionnaires typically produced for each trial and rarely validated. The healthcare module of a generic, modular resource-use measure, designed for collecting self-report resource-utilisation data, has recently been developed in the UK. The objective of this research is to identify and prioritise items for new, bolt-on modules, covering informal care, social care and personal expenses incurred due to health and care needs.

METHODS

Identification and prioritisation, conducted between April and December 2021, involved a rapid review of questionnaires included in the Database of Instruments for Resource Use Measurement and economic evaluations published from 2011 to 2021 to identify candidate items, an online survey of UK-based social care professionals to identify omitted social care items and focus groups with UK-based health economists and UK-based people who access social care services either for themselves or as carers to prioritise items.

RESULTS

The review identified 203 items. Over half of the 24 survey respondents reported no missing items. Five academic health economists and four people who access social care services participated in focus groups. Feedback shaped the social and informal care modules and indicated that no specific personal expenses were essential to collect in all trials. Aids/adaptations were highlighted as costly personal expenses when relevant; therefore, the personal expenses module was narrowed to aids/adaptations only.

CONCLUSION

Draft informal care, social care and aids/adaptations modules were developed, ready for further testing.

Development of a brief, generic, modular resource-use measure (ModRUM): piloting with patients

BACKGROUND

Bespoke self-report resource-use measures (RUMs) are commonly developed or adapted for each new randomised controlled trial. Consequently, RUMs lack standardisation and validation is rarely conducted. A new generic RUM, ModRUM, has been developed using a rigorous process, including consultation with health economists and patients. ModRUM includes a concise core healthcare module, designed to be included in all trials, and depth-adding questions, which can replace or be added to core questions as needed. Modules covering other sectors are under development. The aim of this study was to test the acceptability, feasibility, and criterion and construct validity of the healthcare module of ModRUM.

METHODS

Patients who had a recent appointment at their GP practice were invited to complete ModRUM (core module or core module with depth questions), a characteristics form and the EQ-5D-5L. Acceptability was assessed via response rates and questionnaire completion time. Feasibility was assessed by reviewing issues observed in participants' responses and question completion rates. Construct validity was tested via hypothesis testing and known-group analyses, using Wilcoxon rank-sum and Kruskal-Wallis tests, and a generalised linear model. Criterion validity was tested by comparing ModRUM results with primary care medical records. Sensitivity, specificity, and agreement using Lin's concordance correlation coefficient (pc) were estimated.

RESULTS

One hundred patients participated from five GP practices in the South-West of England. Acceptability was higher for the core module (20% versus 10% response rate). Question completion rates were high across both versions (> 90%). Some support was observed for construct validity, with results suggesting that healthcare costs differ dependent on the number of long-term conditions (p < 0.05) and are negatively associated with health-related quality of life (p < 0.01). Sensitivity was high for all questions (> 0.83), while specificity varied (0.33-0.88). There was a good level of agreement for GP contacts and costs, and prescribed medication costs (pc > 0.6).

CONCLUSION

This study provided preliminary evidence of the acceptability, feasibility, and criterion and construct validity of ModRUM. Further testing is required within trials and with groups that were less well represented in this study.

Impact of PSA testing on secondary care costs in England and Wales: estimates from the Cluster randomised triAl of PSA testing for Prostate cancer (CAP)

BACKGROUND

Screening men for prostate cancer using prostate-specific antigen (PSA) testing remains controversial. We aimed to estimate the likely budgetary impact on secondary care in England and Wales to inform screening decision makers.

METHODS

The Cluster randomised triAl of PSA testing for Prostate cancer study (CAP) compared a single invitation to men aged 50-69 for a PSA test with usual care (no screening). Routinely collected hospital care data were obtained for all men in CAP, and NHS reference costs were mapped to each event via Healthcare Resource Group (HRG) codes. Secondary-care costs per man per year were calculated, and cost differences (and population-level estimates) between arms were derived annually for the first five years following randomisation.

RESULTS

In the first year post-randomisation, secondary-care costs averaged across all men (irrespective of a prostate cancer diagnosis) in the intervention arm (n = 189279) were £44.80 (95% confidence interval: £18.30-£71.30) higher than for men in the control arm (n = 219357). Extrapolated to a population level, the introduction of a single PSA screening invitation could lead to additional secondary care costs of £314 million.

CONCLUSIONS

Introducing a single PSA screening test for men aged 50-69 across England and Wales could lead to very high initial secondary-care costs.

Development of an Instrument for the Assessment of Health-Related Multi-sectoral Resource Use in Europe: The PECUNIA RUM

BACKGROUND

Measuring objective resource-use quantities is important for generating valid cost estimates in economic evaluations. In the absence of acknowledged guidelines, measurement methods are often chosen based on practicality rather than methodological evidence. Furthermore, few resource-use measurement (RUM) instruments focus on the measurement of resource use in multiple societal sectors and their development process is rarely described. Thorn and colleagues proposed a stepwise approach to the development of RUM instruments, which has been used for developing cost questionnaires for specific trials. However, it remains unclear how this approach can be translated into practice and whether it is applicable to the development of generic self-reported RUM instruments and instruments measuring resource use in multiple sectors. This study provides a detailed description of the practical application of this stepwise approach to the development of a multi-sectoral RUM instrument developed within the ProgrammE in Costing, resource use measurement and outcome valuation for Use in multi-sectoral National and International health economic evaluAtions (PECUNIA) project.

METHODS

For the development of the PECUNIA RUM, the methodological approach was based on best practice guidelines. The process included six steps, including the definition of the instrument attributes, identification of cost-driving elements in each sector, review of methodological literature and development of a harmonized cross-sectorial approach, development of questionnaire modules and their subsequent harmonization.

RESULTS

The selected development approach was, overall, applicable to the development of the PECUNIA RUM. However, due to the complexity of the development of a multi-sectoral RUM instrument, additional steps such as establishing a uniform methodological basis, harmonization of questionnaire modules and involvement of a broader range of stakeholders (healthcare professionals, sector-specific experts, health economists) were needed.

CONCLUSION

This is the first study that transparently describes the development process of a generic multi-sectoral RUM instrument in health economics and provides insights into the methodological aspects and overall validity of its development process.

Commentary/Response: Economic evidence should be routinely collected and reported for studies of intervention effectiveness in mental health. A commentary on Vartiainen et al. (2022)

Anxiety and related disorders are increasingly widespread amongst children and adolescents. Preventing mental health disorders from developing has the potential to realise long-term benefits for children and adolescents. In their paper, 'Economic evidence of preventive interventions for anxiety disorders in children and adolescents-a systematic review', Vartiainen et al. conducted a systematic review to examine economic evidence of interventions for the primary prevention of anxiety disorders in children and adolescents, under 18 years of age. Five articles were eligible for inclusion in the review, of which two were model-based economic evaluations and three conducted alongside randomised controlled trials (RCTs). All five papers used either a cost-effectiveness analysis (CEA) or cost-utility analysis (CUA) as their main analysis. Vartiainen et al. concluded that, due to the small number of studies and relatively small sample sizes, the evidence for the cost-effectiveness of anxiety prevention interventions is weak. In this commentary, the challenges of conducting economic evaluations for prevention interventions are briefly outlined and Vartiainen et al.'s findings are discussed in the context of two further reviews of economic studies, published in 2021. The first focuses on the prevention of anxiety and depression in children and young people and the second takes a broader perspective and also includes interventions for mental health promotion. Both additional reviews note the small number of published economic evaluations, and all three reviews are united in their call for economic evaluations to be conducted alongside all future mental health prevention intervention trials.

Aspects and Challenges of Resource Use Measurement in Health Economics: Towards a Comprehensive Measurement Framework

BACKGROUND

While the methods for conducting health economics research in general are improving, current guidelines provide limited guidance regarding resource use measurement (RUM). Consequently, a variety of methods exists, yet there is no overview of aspects to consider when deciding on the most appropriate RUM methodology. Therefore, this study aims to (1) identify and categorize existing knowledge regarding aspects of RUM, and (2) develop a framework that provides a comprehensive overview of methodological aspects regarding RUM.

METHODS

Relevant articles were identified by enrolling a search string in six databases and handsearching the DIRUM database. Included articles were descriptively reviewed and served as input for a comprehensive framework. Health economics experts were involved during the process to establish the framework's face validity.

RESULTS

Forty articles were included in the scoping review. The RUM framework consists of four methodological RUM domains: 'Whom to measure', addressing whom to ask and whom to measure; 'How to measure', addressing the different approaches of measurement; 'How often to measure', addressing recall period and measurement patterns; and 'Additional considerations', which covers additional aspects that are essential for further refining the methodologies for measurement. Evidence retrieved from the scoping review was categorized according to these domains.

CONCLUSION

This study clustered the aspects of RUM methodology in health economics into a comprehensive framework. The results may guide health economists in their decision making regarding the selection of appropriate RUM methods and developing instruments for RUM. Furthermore, policy makers may use these findings to review study results from an evidence-based perspective.

School-based interventions to prevent anxiety, depression and conduct disorder in children and young people: a systematic review and network meta-analysis

Background

Schools in the UK increasingly have to respond to anxiety, depression and conduct disorder as key causes of morbidity in children and young people.

Objective

The objective was to assess the comparative effectiveness of educational setting-based interventions for the prevention of anxiety, depression and conduct disorder in children and young people.

Design

This study comprised a systematic review, a network meta-analysis and an economic evaluation.

Data sources

The databases MEDLINE, EMBASE™ (Elsevier, Amsterdam, the Netherlands), PsycInfo® (American Psychological Association, Washington, DC, USA) and Cochrane Central Register of Controlled Trials (CENTRAL) were searched to 4 April 2018, and the NHS Economic Evaluation Database (NHS EED) was searched on 22 May 2019 for economic evaluations. No language or date filters were applied.

Main outcomes

The main outcomes were post-intervention self-reported anxiety, depression or conduct disorder symptoms.

Review methods

Randomised/quasi-randomised trials of universal or targeted interventions for the prevention of anxiety, depression or conduct disorder in children and young people aged 4–18 years were included. Screening was conducted independently by two reviewers. Data extraction was conducted by one reviewer and checked by a second. Intervention- and component-level network meta-analyses were conducted in OpenBUGS. A review of the economic literature and a cost–consequence analysis were conducted.

Results

A total of 142 studies were included in the review, and 109 contributed to the network meta-analysis. Of the 109 studies, 57 were rated as having an unclear risk of bias for random sequence generation and allocation concealment. Heterogeneity was moderate. In universal secondary school settings, mindfulness/relaxation interventions [standardised mean difference (SMD) –0.65, 95% credible interval (CrI) –1.14 to –0.19] and cognitive–behavioural interventions (SMD –0.15, 95% CrI –0.34 to 0.04) may be effective for anxiety. Cognitive–behavioural interventions incorporating a psychoeducation component may be effective (SMD –0.30, 95% CrI –0.59 to –0.01) at preventing anxiety immediately post intervention. There was evidence that exercise was effective in preventing anxiety in targeted secondary school settings (SMD –0.47, 95% CrI –0.86 to –0.09). There was weak evidence that cognitive–behavioural interventions may prevent anxiety in universal (SMD –0.07, 95% CrI –0.23 to 0.05) and targeted (SMD –0.38, 95% CrI –0.84 to 0.07) primary school settings. There was weak evidence that cognitive–behavioural (SMD –0.04, 95% CrI –0.16 to 0.07) and cognitive–behavioural + interpersonal therapy (SMD –0.18, 95% CrI –0.46 to 0.08) may be effective in preventing depression in universal secondary school settings. Third-wave (SMD –0.35, 95% CrI –0.70 to 0.00) and cognitive–behavioural interventions (SMD –0.11, 95% CrI –0.28 to 0.05) incorporating a psychoeducation component may be effective at preventing depression immediately post intervention. There was no evidence of intervention effectiveness in targeted secondary, targeted primary or universal primary school settings post intervention. The results for university settings were unreliable because of inconsistency in the network meta-analysis. A narrative summary was reported for five conduct disorder prevention studies, all in primary school settings. None reported the primary outcome at the primary post-intervention time point. The economic evidence review reported heterogeneous findings from six studies. Taking the perspective of a single school budget and based on cognitive–behavioural therapy intervention costs in universal secondary school settings, the cost–consequence analysis estimated an intervention cost of £43 per student.

Limitations

The emphasis on disorder-specific prevention excluded broader mental health interventions and restricted the number of eligible conduct disorder prevention studies. Restricting the study to interventions delivered in the educational setting may have limited the number of eligible university-level interventions.

Conclusions

There was weak evidence of the effectiveness of school-based, disorder-specific prevention interventions, although effects were modest and the evidence not robust. Cognitive–behavioural therapy-based interventions may be more effective if they include a psychoeducation component.

Future work

Future trials for prevention of anxiety and depression should evaluate cognitive–behavioural interventions with and without a psychoeducation component, and include mindfulness/relaxation or exercise comparators, with sufficient follow-up. Cost implications must be adequately measured.

Study registration

This study is registered as PROSPERO CRD42016048184.

Funding

This project was funded by the National Institute for Health Research (NIHR) Public Health Research programme and will be published in full in Public Health Research; Vol. 9, No. 8. See the NIHR Journals Library website for further project information.

Development of a brief, generic, modular resource-use measure (ModRUM): cognitive interviews with patients

BACKGROUND

Self-report resource-use measures (RUMs) are often used to collect healthcare use data from participants in healthcare studies. However, RUMs are typically adapted from existing measures on a study-by-study basis, resulting in a lack of standardisation which limits comparability across studies. Psychometric testing of RUMs is rarely conducted. This paper reports on cognitive interviews with patients to test the content validity and acceptability of a new RUM (ModRUM). ModRUM is a brief, generic RUM with a core module on healthcare use and questions/modules to increase depth and breadth.

METHODS

A purposeful sampling strategy with maximum variation was used to recruit patients from primary care to participate in "think-aloud" interviews with retrospective probing. Participants verbalised their thought processes as they completed ModRUM, which allowed errors (issues with completion) to be identified. The interviewer asked follow-up and probing questions to investigate errors, clarity and acceptability. Interviews were audio-recorded and transcribed verbatim. Research team members independently scored transcripts to identify errors in comprehension, recall, judgement and response. Members met to agree on final scores. Interview transcripts were analysed qualitatively using techniques of constant comparison, to identify common themes and ideas for improvement. Data collection and analysis were performed concurrently and in rounds.

RESULTS

Twenty participants were interviewed between December 2019 and March 2020. Interviews were conducted in three rounds, with revisions made iteratively and in response to interview findings. Seven participants completed the core module and 13 completed the core module plus depth questions. Of 71 issues, 28 were in comprehension, 14 in retrieval, 10 in judgement, 18 in response and 1 uncategorised. Most issues (21 issues by 2 participants) were due to participants including family healthcare use. Other issues included using incorrect recall periods (5 issues) and overlooking questions leading to missing responses (9 issues). Common participant suggestions included highlighting important details and providing additional definition or examples for some terms. The length, content and layout were acceptable to most participants.

CONCLUSIONS

A generic RUM is needed to increase study comparability. RUM development requires thorough testing to demonstrate and enhance validity. Cognitive interviewing has demonstrated the acceptability and content validity of ModRUM.

Content of Health Economics Analysis Plans (HEAPs) for Trial-Based Economic Evaluations: Expert Delphi Consensus Survey

OBJECTIVES

Health economics analysis plans (HEAPs) currently lack consistency, with uncertainty surrounding appropriate content. We aimed to develop a list of essential items that should be included in HEAPs for economic evaluations conducted alongside randomized trials.

METHODS

A list of potential items for inclusion was developed by examining existing HEAPs. An electronic Delphi survey was conducted among professional health economists. Respondents were asked to rate potential items from 1 (least important) to 9 (most important), suggest additional items, and comment on proposed items (round 1). A second survey (round 2) was emailed to participants, including the participant's own scores from round 1 along with summary results from the whole panel; participants were asked to rerate each item. Consensus criteria for inclusion in the final list were predefined as >70% of participants rating an item 7-9 and <15% rating it 1-3 after round 2. A final item selection meeting was held to scrutinize the results and adjudicate on items lacking consensus.

RESULTS

62 participants completed round 1 of the survey. The initial list included 72 potential items; all 72 were carried forward to round 2, and no new items were added. 48 round 1 respondents (77.4%) completed round 2 and reached consensus on 53 items. At the final meeting, the expert panel (n = 9) agreed that 58 items should be included in the essential list, moved 9 items to an optional list, and dropped 5 items.

CONCLUSIONS

Via expert consensus opinion, this study identified 58 items that are considered essential in a HEAP.

The ProtecT randomised trial cost-effectiveness analysis comparing active monitoring, surgery, or radiotherapy for prostate cancer

BACKGROUND

There is limited evidence relating to the cost-effectiveness of treatments for localised prostate cancer.

METHODS

The cost-effectiveness of active monitoring, surgery, and radiotherapy was evaluated within the Prostate Testing for Cancer and Treatment (ProtecT) randomised controlled trial from a UK NHS perspective at 10 years' median follow-up. Prostate cancer resource-use collected from hospital records and trial participants was valued using UK reference-costs. QALYs (quality-adjusted-life-years) were calculated from patient-reported EQ-5D-3L measurements. Adjusted mean costs, QALYs, and incremental cost-effectiveness ratios were calculated; cost-effectiveness acceptability curves and sensitivity analyses addressed uncertainty; subgroup analyses considered age and disease-risk.

RESULTS

Adjusted mean QALYs were similar between groups: 6.89 (active monitoring), 7.09 (radiotherapy), and 6.91 (surgery). Active monitoring had lower adjusted mean costs (£5913) than radiotherapy (£7361) and surgery (£7519). Radiotherapy was the most likely (58% probability) cost-effective option at the UK NICE willingness-to-pay threshold (£20,000 per QALY). Subgroup analyses confirmed radiotherapy was cost-effective for older men and intermediate/high-risk disease groups; active monitoring was more likely to be the cost-effective option for younger men and low-risk groups.

CONCLUSIONS

Longer follow-up and modelling are required to determine the most cost-effective treatment for localised prostate cancer over a man's lifetime.

TRIAL REGISTRATION

Current Controlled Trials number, ISRCTN20141297: http://isrctn.org (14/10/2002); ClinicalTrials.gov number, NCT02044172: http://www.clinicaltrials.gov (23/01/2014).

Core Items for a Standardized Resource Use Measure: Expert Delphi Consensus Survey

BACKGROUND

Resource use measurement by patient recall is characterized by inconsistent methods and a lack of validation. A validated standardized resource use measure could increase data quality, improve comparability between studies, and reduce research burden.

OBJECTIVES

To identify a minimum set of core resource use items that should be included in a standardized adult instrument for UK health economic evaluation from a provider perspective.

METHODS

Health economists with experience of UK-based economic evaluations were recruited to participate in an electronic Delphi survey. Respondents were asked to rate 60 resource use items (e.g., medication names) on a scale of 1 to 9 according to the importance of the item in a generic context. Items considered less important according to predefined consensus criteria were dropped and a second survey was developed. In the second round, respondents received the median score and their own score from round 1 for each item alongside summarized comments and were asked to rerate items. A final project team meeting was held to determine the recommended core set.

RESULTS

Forty-five participants completed round 1. Twenty-six items were considered less important and were dropped, 34 items were retained for the second round, and no new items were added. Forty-two respondents (93.3%) completed round 2, and greater consensus was observed. After the final meeting, 10 core items were selected, with further items identified as suitable for "bolt-on" questionnaire modules.

CONCLUSIONS

The consensus on 10 items considered important in a generic context suggests that a standardized instrument for core resource use items is feasible.

Effect of a Low-Intensity PSA-Based Screening Intervention on Prostate Cancer Mortality: The CAP Randomized Clinical Trial

Importance

Prostate cancer screening remains controversial because potential mortality or quality-of-life benefits may be outweighed by harms from overdetection and overtreatment.

Objective

To evaluate the effect of a single prostate-specific antigen (PSA) screening intervention and standardized diagnostic pathway on prostate cancer-specific mortality.

Design, Setting, and Participants

The Cluster Randomized Trial of PSA Testing for Prostate Cancer (CAP) included 419 582 men aged 50 to 69 years and was conducted at 573 primary care practices across the United Kingdom. Randomization and recruitment of the practices occurred between 2001 and 2009; patient follow-up ended on March 31, 2016.

Intervention

An invitation to attend a PSA testing clinic and receive a single PSA test vs standard (unscreened) practice.

Main Outcomes and Measures

Primary outcome: prostate cancer-specific mortality at a median follow-up of 10 years. Prespecified secondary outcomes: diagnostic cancer stage and Gleason grade (range, 2-10; higher scores indicate a poorer prognosis) of prostate cancers identified, all-cause mortality, and an instrumental variable analysis estimating the causal effect of attending the PSA screening clinic.

Results

Among 415 357 randomized men (mean [SD] age, 59.0 [5.6] years), 189 386 in the intervention group and 219 439 in the control group were included in the analysis (n = 408 825; 98%). In the intervention group, 75 707 (40%) attended the PSA testing clinic and 67 313 (36%) underwent PSA testing. Of 64 436 with a valid PSA test result, 6857 (11%) had a PSA level between 3 ng/mL and 19.9 ng/mL, of whom 5850 (85%) had a prostate biopsy. After a median follow-up of 10 years, 549 (0.30 per 1000 person-years) died of prostate cancer in the intervention group vs 647 (0.31 per 1000 person-years) in the control group (rate difference, -0.013 per 1000 person-years [95% CI, -0.047 to 0.022]; rate ratio [RR], 0.96 [95% CI, 0.85 to 1.08]; P = .50). The number diagnosed with prostate cancer was higher in the intervention group (n = 8054; 4.3%) than in the control group (n = 7853; 3.6%) (RR, 1.19 [95% CI, 1.14 to 1.25]; P < .001). More prostate cancer tumors with a Gleason grade of 6 or lower were identified in the intervention group (n = 3263/189 386 [1.7%]) than in the control group (n = 2440/219 439 [1.1%]) (difference per 1000 men, 6.11 [95% CI, 5.38 to 6.84]; P < .001). In the analysis of all-cause mortality, there were 25 459 deaths in the intervention group vs 28 306 deaths in the control group (RR, 0.99 [95% CI, 0.94 to 1.03]; P = .49). In the instrumental variable analysis for prostate cancer mortality, the adherence-adjusted causal RR was 0.93 (95% CI, 0.67 to 1.29; P = .66).

Conclusions and Relevance

Among practices randomized to a single PSA screening intervention vs standard practice without screening, there was no significant difference in prostate cancer mortality after a median follow-up of 10 years but the detection of low-risk prostate cancer cases increased. Although longer-term follow-up is under way, the findings do not support single PSA testing for population-based screening.

Trial Registration

ISRCTN Identifier: ISRCTN92187251.

Validating the use of Hospital Episode Statistics data and comparison of costing methodologies for economic evaluation: an end-of-life case study from the Cluster randomised triAl of PSA testing for Prostate cancer (CAP)

OBJECTIVES

To evaluate the accuracy of routine data for costing inpatient resource use in a large clinical trial and to investigate costing methodologies.

DESIGN

Final-year inpatient cost profiles were derived using (1) data extracted from medical records mapped to the National Health Service (NHS) reference costs via service codes and (2) Hospital Episode Statistics (HES) data using NHS reference costs. Trust finance departments were consulted to obtain costs for comparison purposes.

SETTING

7 UK secondary care centres.

POPULATION

A subsample of 292 men identified as having died at least a year after being diagnosed with prostate cancer in Cluster randomised triAl of PSA testing for Prostate cancer (CAP), a long-running trial to evaluate the effectiveness and cost-effectiveness of prostate-specific antigen (PSA) testing.

RESULTS

Both inpatient cost profiles showed a rise in costs in the months leading up to death, and were broadly similar. The difference in mean inpatient costs was £899, with HES data yielding ∼8% lower costs than medical record data (differences compatible with chance, p=0.3). Events were missing from both data sets. 11 men (3.8%) had events identified in HES that were all missing from medical record review, while 7 men (2.4%) had events identified in medical record review that were all missing from HES. The response from finance departments to requests for cost data was poor: only 3 of 7 departments returned adequate data sets within 6 months.

CONCLUSIONS

Using HES routine data coupled with NHS reference costs resulted in mean annual inpatient costs that were very similar to those derived via medical record review; therefore, routinely available data can be used as the primary method of costing resource use in large clinical trials. Neither HES nor medical record review represent gold standards of data collection. Requesting cost data from finance departments is impractical for large clinical trials.

TRIAL REGISTRATION NUMBER

ISRCTN92187251; Pre-results.

Validation of the Hospital Episode Statistics Outpatient Dataset in England

OBJECTIVES

The Hospital Episode Statistics (HES) dataset is a source of administrative 'big data' with potential for costing purposes in economic evaluations alongside clinical trials. This study assesses the validity of coverage in the HES outpatient dataset.

METHODS

Men who died of, or with, prostate cancer were selected from a prostate-cancer screening trial (CAP, Cluster randomised triAl of PSA testing for Prostate cancer). Details of visits that took place after 1/4/2003 to hospital outpatient departments for conditions related to prostate cancer were extracted from medical records (MR); these appointments were sought in the HES outpatient dataset based on date. The matching procedure was repeated for periods before and after 1/4/2008, when the HES outpatient dataset was accredited as a national statistic.

RESULTS

4922 outpatient appointments were extracted from MR for 370 men. 4088 appointments recorded in MR were identified in the HES outpatient dataset (83.1%; 95% confidence interval [CI] 82.0-84.1). For appointments occurring prior to 1/4/2008, 2195/2755 (79.7%; 95% CI 78.2-81.2) matches were observed, while 1893/2167 (87.4%; 95% CI 86.0-88.9) appointments occurring after 1/4/2008 were identified (p for difference <0.001). 215/370 men (58.1%) had at least one appointment in the MR review that was unmatched in HES, 155 men (41.9%) had all their appointments identified, and 20 men (5.4%) had no appointments identified in HES.

CONCLUSIONS

The HES outpatient dataset appears reasonably valid for research, particularly following accreditation. The dataset may be a suitable alternative to collecting MR data from hospital notes within a trial, although caution should be exercised with data collected prior to accreditation.

Methodological developments in randomized controlled trial-based economic evaluations

Economic evaluation is a key contributor to decision making in health care, and it is important that it is carried out as effectively and reliably as possible. Studies carried out alongside randomised controlled trials are required to contribute real-world evidence to the decision-making process. However, the requirement that resource use be measured as well as effectiveness data within a trial results in additional complexity for trialists, and there are a number of methodological areas in which improvement is needed. This article reviews the literature in methodological work carried out to inform economic evaluation studies conducted alongside randomised controlled trials. Recent advances in areas including overall trial design, measuring resource use, measuring outcomes and reporting economic evaluations are discussed.

Resource-use measurement based on patient recall: issues and challenges for economic evaluation

Accurate resource-use measurement is challenging within an economic evaluation, but is a fundamental requirement for estimating efficiency. Considerable research effort has been concentrated on the appropriate measurement of outcomes and the policy implications of economic evaluation, while methods for resource-use measurement have been relatively neglected. Recently, the Database of Instruments for Resource Use Measurement (DIRUM) was set up at http://www.dirum.org to provide a repository where researchers can share resource-use measures and methods. A workshop to discuss the issues was held at the University of Birmingham in October 2011. Based on material presented at the workshop, this article highlights the state of the art of UK instruments for resource-use data collection based on patient recall. We consider methodological issues in the design and analysis of resource-use instruments, and the challenges associated with designing new questionnaires. We suggest a method of developing a good practice guideline, and identify some areas for future research. Consensus amongst health economists has yet to be reached on many aspects of resource-use measurement. We argue that researchers should now afford costing methodologies the same attention as outcome measurement, and we hope that this Current Opinion article will stimulate a debate on methods of resource-use data collection and establish a research agenda to improve the precision and accuracy of resource-use estimates.

Timely and complete publication of economic evaluations alongside randomized controlled trials

BACKGROUND AND AIMS

Little is known about the extent and nature of publication bias in economic evaluations. Our objective was to determine whether economic evaluations are subject to publication bias by considering whether economic data are as likely to be reported, and reported as promptly, as effectiveness data.

METHODS

Trials that intended to conduct an economic analysis and ended before 2008 were identified in the International Standard Randomised Controlled Trial Number (ISRCTN) register; a random sample of 100 trials was retrieved. Fifty comparator trials were randomly drawn from those not identified as intending to conduct an economic study. The trial start and end dates, estimated sample size and funder type were extracted. For trials planning economic evaluations, effectiveness and economic publications were sought; publication dates and journal impact factors were extracted. Effectiveness abstracts were assessed for whether they reached a firm conclusion that one intervention was most effective. Primary investigators were contacted about reasons for non-publication of results, or reasons for differential publication strategies for effectiveness and economic results.

RESULTS

Trials planning an economic study were more likely to be funded by government (p = 0.01) and larger (p = 0.003) than other trials. The trials planning an economic evaluation had a mean of 6.5 (range 2.7-13.2) years since the trial end in which to publish their results. Effectiveness results were reported by 70 %, while only 43 % published economic evaluations (p < 0.001). Reasons for non-publication of economic results included the intervention being ineffective, and staffing issues. Funding source, time since trial end and length of study were not associated with a higher probability of publishing the economic evaluation. However, studies that were small or of unknown size were significantly less likely to publish economic evaluations than large studies (p < 0.001). The authors' confidence in labelling one intervention clearly most effective did not affect the probability of publication. The mean time to publication was 0.7 years longer for cost-effectiveness data than for effectiveness data where both were published (p = 0.001). The median journal impact factor was 1.6 points higher for effectiveness publications than for the corresponding economic publications (p = 0.01). Reasons for publishing in different journals included editorial decision making and the additional time that economic evaluation takes to conduct.

CONCLUSIONS

Trials that intend to conduct an economic analysis are less likely to report economic data than effectiveness data. Where economic results do appear, they are published later, and in journals with lower impact factors. These results suggest that economic output may be more susceptible than effectiveness data to publication bias. Funders, grant reviewers and trialists themselves should ensure economic evaluations are prioritized and adequately staffed to avoid potential problems with bias.