Can randomised trials rely on existing electronic data? A feasibility study to explore the value of routine data in health technology assessment

Insights from UKCTOCS for design, conduct and analyses of large randomised controlled trials

Abstract

Randomised controlled trials are challenging to deliver. There is a constant need to review and refine recruitment and implementation strategies if they are to be completed on time and within budget. We present the strategies adopted in the United Kingdom Collaborative Trial of Ovarian Cancer Screening, one of the largest individually randomised controlled trials in the world. The trial recruited over 202,000 women (2001-5) and delivered over 670,000 annual screens (2001-11) and over 3 million women-years of follow-up (2001-20). Key to the successful completion were the involvement of senior investigators in the day-to-day running of the trial, proactive trial management and willingness to innovate and use technology. Our underlying ethos was that trial participants should always be at the centre of all our processes. We ensured that they were able to contact either the site or the coordinating centre teams for clarifications about their results, for follow-up and for rescheduling of appointments. To facilitate this, we shared personal identifiers (with consent) with both teams and had dedicated reception staff at both site and coordinating centre. Key aspects were a comprehensive online trial management system which included an electronic data capture system (resulting in an almost paperless trial), biobanking, monitoring and project management modules. The automation of algorithms (to ascertain eligibility and classify results and ensuing actions) and processes (scheduling of appointments, printing of letters, etc.) ensured the protocol was closely followed and timelines were met. Significant engagement with participants ensured retention and low rates of complaints. Our solutions to the design, conduct and analyses issues we faced are highly relevant, given the renewed focus on trials for early detection of cancer.

Future work

There is a pressing need to increase the evidence base to support decision making about all aspects of trial methodology.

Trial registration

ISRCTN-22488978; ClinicalTrials.gov-NCT00058032.

Funding

This article presents independent research funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme as award number 16/46/01. The long-term follow-up UKCTOCS (2015 20) was supported by National Institute for Health and Care Research (NIHR HTA grant 16/46/01), Cancer Research UK, and The Eve Appeal. UKCTOCS (2001-14) was funded by the MRC (G9901012 and G0801228), Cancer Research UK (C1479/A2884), and the UK Department of Health, with additional support from The Eve Appeal. Researchers at UCL were supported by the NIHR UCL Hospitals Biomedical Research Centre and by the MRC Clinical Trials Unit at UCL core funding (MC_UU_00004/09, MC_UU_00004/08, MC_UU_00004/07). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, or the UK Department of Health and Social Care.

Augmenting clinical trial economic analysis by linking cancer trial data to administrative data: current landscape and future opportunities

BACKGROUND

Economic analyses based on clinical trial data are costly and time consuming, and alternative methods for performing economic analyses should be explored.

OBJECTIVE AND METHODS

In this perspective, we examine the emerging role of administrative data for economic analyses in cancer.

RESULTS

Compared with routinely collected clinical trial data, routinely collected administrative data have several strengths including high capture rates for healthcare encounters, less resource utilisation, low rates of misclassification, long follow-up periods and the opportunity to collect data points not traditionally captured in clinical trials. However, there are also limitations including the need for accurate data linkage across multiple databases and systems, the costs and time associated with data linkage, the potential time lag between trial data collection and the availability of administrative data, and limited data on quality of life, toxicity and indirect costs. In this perspective, we identify important barriers and potential solutions to performing economic analyses for oncology using administrative data, and outline strategies to increase research in this field.

CONCLUSION

The use of routinely collected administrative data sets for economic analyses of clinical trials presents a unique opportunity that could complement and validate economic analyses based on trial-level data.

Using routinely recorded data in a UK RCT: a comparison to standard prospective data collection methods

BACKGROUND

Routinely recorded data held in electronic health records can be used to inform the conduct of randomised controlled trials (RCTs). However, limitations with access and accuracy have been identified.

OBJECTIVE

Using epilepsy as an exemplar condition, we assessed the attributes and agreement of routinely recorded data compared to data collected using case report forms in a UK RCT assessing antiepileptic drug treatments for individuals newly diagnosed with epilepsy.

METHODS

The case study RCT is the Standard and New Antiepileptic Drugs II (SANAD II) trial, a pragmatic, UK multicentre RCT assessing the clinical and cost-effectiveness of antiepileptic drugs as treatments for epilepsy. Ninety-eight of 470 eligible participants provided consent for access to routinely recorded secondary care data that were retrieved from NHS Digital Hospital Episode Statistics (N=71) and primary and secondary care data from The Secure Anonymised Information Linkage Databank (N=27). We assessed data items relevant to the identification of individuals eligible for inclusion in SANAD II, baseline and follow-up visits. The attributes of routinely recorded data were assessed including the degree of missing data. The agreement between routinely recorded data and data collected on case report forms in SANAD II was assessed using calculation of Cohen's kappa for categorical data and construction of Bland-Altman plots for continuous data.

RESULTS

There was a significant degree of missing data in the routine record for 15 of the 20 variables assessed, including all clinical variables. Agreement was poor for the majority of comparisons, including the assessments of seizure occurrence and adverse events. For example, only 23/62 (37%) participants had a date of first-ever seizure identified in routine datasets. Agreement was satisfactory for the date of prescription of antiepileptic drugs and episodes of healthcare resource use.

CONCLUSIONS

There are currently significant limitations preventing the use of routinely recorded data for participant identification and assessment of clinical outcomes in epilepsy, and potentially other chronic conditions. Further research is urgently required to assess the attributes, agreement, additional benefits, cost-effectiveness and 'optimal mix' of routinely recorded data compared to data collected using standard methods such as case report forms at clinic visits for people with epilepsy.

TRIAL REGISTRATION

Standard and New Antiepileptic Drugs II (SANAD II (EudraCT No: 2012-001884-64, registered 05/09/2012; ISRCTN Number: ISRCTN30294119 , registered 03/07/2012)).

Assessing the use of surveillance data to estimate the impact of prevention interventions on HIV incidence in cluster-randomized controlled trials

BACKGROUND

In cluster-randomized controlled trials (C-RCTs) of HIV prevention strategies, HIV incidence is expensive to measure directly. Surveillance data on HIV diagnoses or viral suppression could provide cheaper incidence estimates. We used mathematical modelling to evaluate whether these measures can replace HIV incidence measurement in C-RCTs.

METHODS

We used a US HIV transmission model to simulate C-RCTs of expanded antiretroviral therapy(ART), pre-exposure prophylaxis(PrEP) and HIV testing, together or alone. We tested whether modelled reductions in total new HIV diagnoses, diagnoses with acute infection, diagnoses with early infection(CD4 > 500 cells/μl), diagnoses adjusted for testing volume, or the proportion virally non-suppressed, reflected HIV incidence reductions.

RESULTS

Over a two-year trial expanding PrEP alone, modelled reductions in total diagnoses underestimated incidence reductions by a median six percentage points(pp), with acceptable variability(95 % credible interval -14,-2pp). For trials expanding HIV testing alone or alongside ART + PrEP, greater, highly variable bias was seen[-20pp(-128,-1) and -30pp(-134,-16), respectively]. Acceptable levels of bias were only seen over longer trial durations when levels of awareness of HIV-positive status were already high. Expanding ART alone, only acute and early diagnoses reductions reflected incidence reduction well, with some bias[-3pp(-6,-1) and -8pp(-16,-3), respectively]. Early and adjusted diagnoses also reliably reflected incidence when scaling up PrEP alone[bias -5pp(-11,1) and 10pp(3,18), respectively]. For trials expanding testing (alone or with ART + PrEP), bias for all measures explored was too variable for them to replace direct incidence measures, unless using diagnoses when HIV status awareness was already high.

CONCLUSIONS

Surveillance measures based on HIV diagnoses may sometimes be adequate surrogates for HIV incidence reduction in C-RCTs expanding ART or PrEP only, if adjusted for bias. However, all surveillance measures explored failed to approximate HIV incidence reductions for C-RCTs expanding HIV testing, unless levels of awareness of HIV-positive status were already high.

Developing a multidisciplinary rehabilitation package following hip fracture and testing in a randomised feasibility study: Fracture in the Elderly Multidisciplinary Rehabilitation (FEMuR)

BACKGROUND

Proximal femoral fracture is a major health problem in old age, with annual UK health and social care costs of £2.3B. Rehabilitation has the potential to maximise functional recovery and maintain independent living, but evidence of clinical effectiveness and cost-effectiveness is lacking.

OBJECTIVES

To develop an enhanced community-based rehabilitation package following surgical treatment for proximal femoral fracture and to assess acceptability and feasibility for a future definitive randomised controlled trial (RCT) and economic evaluation.

DESIGN

Phase I - realist review, survey and focus groups to develop the rehabilitation package. Phase II - parallel-group, randomised (using a dynamic adaptive algorithm) feasibility study with focus groups and an anonymised cohort study.

SETTING

Recruitment was from orthopaedic wards of three acute hospitals in the Betsi Cadwaladr University Health Board, North Wales. The intervention was delivered in the community following hospital discharge.

PARTICIPANTS

Older adults (aged ≥ 65 years) who had received surgical treatment for hip fracture, lived independently prior to fracture, had mental capacity (assessed by the clinical team) and received rehabilitation in the North Wales area.

INTERVENTIONS

Participants received usual care (control) or usual care plus an enhanced rehabilitation package (intervention). Usual care was variable and consisted of multidisciplinary rehabilitation delivered by the acute hospital, community hospital and community services depending on need and availability. The intervention was designed to enhance rehabilitation by improving patients' self-efficacy and increasing the amount and quality of patients' practice of physical exercise and activities of daily living. It consisted of a patient-held information workbook, a goal-setting diary and six additional therapy sessions.

MAIN OUTCOME MEASURES

The primary outcome measure was the Barthel Activities of Daily Living (BADL) index. The secondary outcome measures included the Nottingham Extended Activities of Daily Living (NEADL) scale, EuroQol-5 Dimensions, ICEpop CAPability measure for Older people, General Self-Efficacy Scale, Falls Efficacy Scale - International (FES-I), Self-Efficacy for Exercise scale, Hospital Anxiety and Depression Scale (HADS) and service use measures. Outcome measures were assessed at baseline and at 3-month follow-up by blinded researchers.

RESULTS

Sixty-two participants were recruited (23% of those who were eligible), 61 were randomised (control, n = 32; intervention, n = 29) and 49 (79%) were followed up at 3 months. Compared with the cohort study, a younger, healthier subpopulation was recruited. There were minimal differences in most outcomes between the two groups, including the BADL index, with an adjusted mean difference of 0.5 (Cohen's d = 0.29). The intervention group showed a medium-sized improvement on the NEADL scale relative to the control group, with an adjusted mean difference between groups of 3.0 (Cohen's d = 0.63). There was a trend for greater improvement in FES-I and HADS in the intervention group, but with small effect sizes, with an adjusted mean difference of 4.2 (Cohen's d = 0.31) and 1.3 (Cohen's d = 0.20), respectively. The cost of delivering the intervention was £231 per patient. There was a possible small relative increase in quality-adjusted life-years in the intervention group. No serious adverse events relating to the intervention were reported.

CONCLUSIONS

Trial methods were feasible in terms of eligibility, recruitment and retention, although recruitment was challenging. The NEADL scale was more responsive than the BADL index, suggesting that the intervention could enable participants to regain better levels of independence compared with usual care. This should be tested in a definitive Phase III RCT. There were two main limitations of the study: the feasibility study lacked power to test for differences between the groups and a ceiling effect was observed in the primary measure.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN22464643.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 21, No. 44. See the NIHR Journals Library for further project information.

Hip fracture in the elderly multidisciplinary rehabilitation (FEMuR) feasibility study: testing the use of routinely collected data for future health economic evaluations

Background

Health economic evaluations rely on the accurate measurement of health service resource use in order to calculate costs. These are usually measured with patient completed questionnaires using instruments such as the Client Service Receipt Inventory (CSRI). These rely on participants' recall and can be burdensome to complete. Health service activity data are routinely captured by electronic databases.The aim was to test methods for obtaining these data and compare with those data collected using the CSRI, within a feasibility study of an enhanced rehabilitation intervention following hip fracture (Fracture in the Elderly Multidisciplinary Rehabilitation: FEMuR).

Methods

Primary care activity including prescribing data was obtained from the Secure Anonymised Information Linkage (SAIL) Databank and secondary care activity (Emergency Department attendances, out-patient visits and in-patient days) directly from Betsi Cadwaladr University Health Board (BCUHB), North Wales, UK. These data were compared with patient responses from the CSRI using descriptive statistics and the intraclass correlation coefficient (ICC).

Results

It was possible to compare health service resource use data for 49 out of 61 participants in the FEMuR study. For emergency department (ED) attendances, records matched in 23 (47%) cases, 21 (43%) over-reported on electronic records compared with CSRI and five participants (10%) under-reported, with an overall ICC of 0.42. For out-patient episodes, records matched in only six cases, 28 participants over-reported on electronic records compared with CSRI and 15 (12%) under-reported, with an overall ICC of only 0.27. For in-patient days, records matched exactly in only five cases (10%), but if an error margin of 7 days was allowed, then agreement rose to 39 (66%) cases, and the overall ICC for all data was 0.88.It was only possible to compare prescribing data for 12 participants. For prescribing data, the SAIL data reported 117 out of 118 items (99%) and the CSRI only 89 (79%) items.

Conclusions

The use of routinely collected data has the potential to improve the efficiency of trials and other studies. Although the methodology to make the data available has been demonstrated, the data obtained was incomplete and the validity of using this method remains to be demonstrated.

Trial registration

Trial registration: ISRCTN22464643 Registered 21 July 2014.

Routine hospital data - is it good enough for trials? An example using England's Hospital Episode Statistics in the SHIFT trial of Family Therapy vs. Treatment as Usual in adolescents following self-harm

BACKGROUND

Use of routine data sources within clinical research is increasing and is endorsed by the National Institute for Health Research to increase trial efficiencies; however there is limited evidence for its use in clinical trials, especially in relation to self-harm. One source of routine data, Hospital Episode Statistics, is collated and distributed by NHS Digital and contains details of admissions, outpatient, and Accident and Emergency attendances provided periodically by English National Health Service hospitals. We explored the reliability and accuracy of Hospital Episode Statistics, compared to data collected directly from hospital records, to assess whether it would provide complete, accurate, and reliable means of acquiring hospital attendances for self-harm - the primary outcome for the SHIFT (Self-Harm Intervention: Family Therapy) trial evaluating Family Therapy for adolescents following self-harm.

METHODS

Participant identifiers were linked to Hospital Episode Statistics Accident and Emergency, and Admissions data, and episodes combined to describe participants' complete hospital attendance. Attendance data were initially compared to data previously gathered by trial researchers from pre-identified hospitals. Final comparison was conducted of subsequent attendances collected through Hospital Episode Statistics and researcher follow-up. Consideration was given to linkage rates; number and proportion of attendances retrieved; reliability of Accident and Emergency, and Admissions data; percentage of self-harm episodes recorded and coded appropriately; and percentage of required data items retrieved.

RESULTS

Participants were first linked to Hospital Episode Statistics with an acceptable match rate of 95%, identifying a total of 341 complete hospital attendances, compared to 139 reported by the researchers at the time. More than double the proportion of Hospital Episode Statistics Accident and Emergency episodes could not be classified in relation to self-harm (75%) compared to 34.9% of admitted episodes, and of overall attendances, 18% were classified as self-harm related and 20% not related, while ambiguity or insufficient information meant 62% were unclassified. Of 39 self-harm-related attendances reported by the researchers, Hospital Episode Statistics identified 24 (62%) as self-harm related while 15 (38%) were unclassified. Based on final data received, 1490 complete hospital attendances were identified and comparison to researcher follow-up found Hospital Episode Statistics underestimated the number of self-harm attendances by 37.2% (95% confidence interval 32.6%-41.9%).

CONCLUSION

Advantages of routine data collection via NHS Digital included the acquisition of more comprehensive and timely trial outcome data, identifying more than double the number of hospital attendances than researchers. Disadvantages included ambiguity in the classification of self-harm relatedness. Our resulting primary outcome data collection strategy used routine data to identify hospital attendances supplemented by targeted researcher data collection for attendances requiring further self-harm classification.

Can learning health systems help organisations deliver personalised care?

There is increasing international policy and clinical interest in developing learning health systems and delivering precision medicine, which it is hoped will help reduce variation in the quality and safety of care, improve efficiency, and lead to increasing the personalisation of healthcare. Although reliant on similar policies, informatics tools, and data science and implementation research capabilities, these two major initiatives have thus far largely progressed in parallel. In this opinion piece, we argue that they should be considered as complementary, synergistic initiatives whereby the creation of learning health systems infrastructure can support and catalyse the delivery of precision medicine that maximises the benefits and minimises the risks associated with treatments for individual patients. We illustrate this synergy by considering the example of treatments for asthma, which is now recognised as an umbrella term for a heterogeneous group of related conditions.

Using routinely recorded data in the UK to assess outcomes in a randomised controlled trial: The Trials of Access

BACKGROUND

In the UK, routinely recorded data may benefit prospective studies including randomised controlled trials (RCTs). In an on-going study, we aim to assess the feasibility of access and agreement of routinely recorded clinical and non-clinical data compared to data collected during a RCT using standard prospective methods. This paper will summarise available UK routinely recorded data sources and discuss our experience with the feasibility of accessing routinely recorded data for participants of a RCT before finally proposing recommendations for improving the access and implementation of routinely recorded data in RCTs.

METHODS

Setting: the case study RCT is the Standard and New Antiepileptic Drugs II (SANAD II) trial, a pragmatic, UK, multicentre, phase IV RCT assessing the clinical and cost-effectiveness of antiepileptic drug treatments for newly diagnosed epilepsy.

PARTICIPANTS

98 participants have provided written consent to permit the request of routinely recorded data. Study procedures: routinely recorded clinical and non-clinical data were identified and data requested through formal applications from available data holders for the duration that participants have been recruited into SANAD II. The feasibility of accessing routinely recorded data during a RCT is assessed and recommendations for improving access proposed.

RESULTS

Secondary-care clinical and socioeconomic data is recorded on a national basis and can be accessed, although there are limitations in the application process. Primary-care data are recorded by a number of organisations on a de-identified basis but access for specific individuals has not been feasible. Access to data recorded by non-clinical sources, including The Department for Work and Pensions and The Driving and Vehicle Licensing Agency, was not successful.

CONCLUSIONS

Recommendations discussed include further research to assess the attributes of routinely recorded data, an assessment of public perceptions and the development of strategies to collaboratively improve access to routinely recorded data for research.

TRIAL REGISTRATION

International Standard Randomised Controlled Trials, ISRCTN30294119 . Registered on 3 July 2012. EudraCT No: 2012-001884-64. Registered on 9 May 2012.

Mid-urethral sling operations for stress urinary incontinence in women

BACKGROUND

Urinary incontinence is a very common and debilitating problem affecting about 50% of women at some point in their lives. Stress urinary incontinence (SUI) is a predominant cause in 30% to 80% of these women imposing significant health and economic burden on society and the women affected. Mid-urethral sling (MUS) operations are a recognised minimally invasive surgical treatment for SUI. MUS involves the passage of a small strip of tape through either the retropubic or obturator space, with entry or exit points at the lower abdomen or groin, respectively. This review does not include single-incision slings.

OBJECTIVES

To assess the clinical effects of mid-urethral sling (MUS) operations for the treatment of SUI, urodynamic stress incontinence (USI) or mixed urinary incontinence (MUI) in women.

SEARCH METHODS

We searched: Cochrane Incontinence Specialised Register (including: CENTRAL, MEDLINE, MEDLINE In-Process, ClinicalTrials.gov) (searched 26 June 2014); Embase Classic (January 1947 to Week 25 2014); WHO ICTRP (searched 30 June 2014); reference lists.

SELECTION CRITERIA

Randomised or quasi-randomised controlled trials amongst women with SUI, USI or MUI, in which both trial arms involve a MUS operation.

DATA COLLECTION AND ANALYSIS

Two review authors independently assessed the methodological quality of potentially eligible studies and extracted data from included trials.

MAIN RESULTS

We included 81 trials that evaluated 12,113 women. We assessed the quality of evidence for outcomes using the GRADE assessment tool; the quality of most outcomes was moderate, mainly due to risk of bias or imprecision.Fifty-five trials with data contributed by 8652 women compared the use of the transobturator route (TOR) and retropubic route (RPR). There is moderate quality evidence that in the short term (up to one year) the rate of subjective cure of TOR and RPR are similar (RR 0.98, 95% CI 0.96 to 1.00; 36 trials, 5514 women; moderate quality evidence) ranging from 62% to 98% in the TOR group, and from 71% to 97% in the RPR group. Short-term objective cure was similar in the TOR and RPR groups (RR 0.98, 95% CI 0.96 to 1.00; 40 trials, 6145 women). Fewer trials reported medium-term (one to five years) and longer-term (over five years) data, but subjective cure was similar between the groups (RR 0.97, 95% CI 0.87 to 1.09; 5 trials, 683 women; low quality evidence; and RR 0.95, 95% CI 0.80 to 1.12; 4 trials, 714 women; moderate quality evidence, respectively). In the long term, subjective cure rates ranged from 43% to 92% in the TOR group, and from 51% to 88% in the RPR group.MUS procedures performed using the RPR had higher morbidity when compared to TOR, though the overall rate of adverse events remained low. The rate of bladder perforation was lower after TOR (0.6% versus 4.5%; RR 0.13, 95% CI 0.08 to 0.20; 40 trials, 6372 women; moderate quality evidence). Major vascular/visceral injury, mean operating time, operative blood loss and length of hospital stay were lower with TOR.Postoperative voiding dysfunction was less frequent following TOR (RR 0.53, 95% CI 0.43 to 0.65; 37 trials, 6200 women; moderate quality evidence). Overall rates of groin pain were higher in the TOR group (6.4% versus 1.3%; RR 4.12, 95% CI 2.71 to 6.27; 18 trials, 3221 women; moderate quality evidence) whereas suprapubic pain was lower in the TOR group (0.8% versus 2.9%; RR 0.29, 95% CI 0.11 to 0.78); both being of short duration. The overall rate of vaginal tape erosion/exposure/extrusion was low in both groups: 24/1000 instances with TOR compared with 21/1000 for RPR (RR 1.13, 95% CI 0.78 to 1.65; 31 trials, 4743 women; moderate quality evidence). There were only limited data to inform the need for repeat incontinence surgery in the long term, but it was more likely in the TOR group than in the RPR group (RR 8.79, 95% CI 3.36 to 23.00; 4 trials, 695 women; low quality evidence).A retropubic bottom-to-top route was more effective than top-to-bottom route for subjective cure (RR 1.10, 95% CI 1.01 to 1.19; 3 trials, 477 women; moderate quality evidence). It incurred significantly less voiding dysfunction, and led to fewer bladder perforations and vaginal tape erosions.Short-and medium-term subjective cure rates between transobturator tapes passed using a medial-to-lateral as opposed to a lateral-to-medial approach were similar (RR 1.00, 95% CI 0.96 to 1.06; 6 trials, 759 women; moderate quality evidence, and RR 1.06, 95% CI 0.91 to 1.23; 2 trials, 235 women; moderate quality evidence). There was moderate quality evidence that voiding dysfunction was more frequent in the medial-to-lateral group (RR 1.74, 95% CI 1.06 to 2.88; 8 trials, 1121 women; moderate quality evidence), but vaginal perforation was less frequent in the medial-to-lateral route (RR 0.25, 95% CI 0.12 to 0.53; 3 trials, 541 women). Due to the very low quality of the evidence, it is unclear whether the lower rates of vaginal epithelial perforation affected vaginal tape erosion (RR 0.42, 95% CI 0.16 to 1.09; 7 trials, 1087 women; very low quality evidence).

AUTHORS' CONCLUSIONS

Mid-urethral sling operations have been the most extensively researched surgical treatment for stress urinary incontinence (SUI) in women and have a good safety profile. Irrespective of the routes traversed, they are highly effective in the short and medium term, and accruing evidence demonstrates their effectiveness in the long term. This review illustrates their positive impact on improving the quality of life of women with SUI. However, a brief economic commentary (BEC) identified three studies suggesting that transobturator may be more cost-effective compared with retropubic. Fewer adverse events occur with employment of a transobturator approach with the exception of groin pain. When comparing transobturator techniques of a medial-to-lateral versus a lateral-to-medial insertion, there is no evidence to support the use of one approach over the other. However, a bottom-to-top route was more effective than top-to-bottom route for retropubic tapes.A salient point illustrated throughout this review is the need for reporting of longer-term outcome data from the numerous existing trials. This would substantially increase the evidence base and provide clarification regarding uncertainties about long-term effectiveness and adverse event profile.

Automated Classification of Selected Data Elements from Free-text Diagnostic Reports for Clinical Research

OBJECTIVES

In the Multiple Myeloma clinical registry at Heidelberg University Hospital, most data are extracted from discharge letters. Our aim was to analyze if it is possible to make the manual documentation process more efficient by using methods of natural language processing for multiclass classification of free-text diagnostic reports to automatically document the diagnosis and state of disease of myeloma patients. The first objective was to create a corpus consisting of free-text diagnosis paragraphs of patients with multiple myeloma from German diagnostic reports, and its manual annotation of relevant data elements by documentation specialists. The second objective was to construct and evaluate a framework using different NLP methods to enable automatic multiclass classification of relevant data elements from free-text diagnostic reports.

METHODS

The main diagnoses paragraph was extracted from the clinical report of one third randomly selected patients of the multiple myeloma research database from Heidelberg University Hospital (in total 737 selected patients). An EDC system was setup and two data entry specialists performed independently a manual documentation of at least nine specific data elements for multiple myeloma characterization. Both data entries were compared and assessed by a third specialist and an annotated text corpus was created. A framework was constructed, consisting of a self-developed package to split multiple diagnosis sequences into several subsequences, four different preprocessing steps to normalize the input data and two classifiers: a maximum entropy classifier (MEC) and a support vector machine (SVM). In total 15 different pipelines were examined and assessed by a ten-fold cross-validation, reiterated 100 times. For quality indication the average error rate and the average F1-score were conducted. For significance testing the approximate randomization test was used.

RESULTS

The created annotated corpus consists of 737 different diagnoses paragraphs with a total number of 865 coded diagnosis. The dataset is publicly available in the supplementary online files for training and testing of further NLP methods. Both classifiers showed low average error rates (MEC: 1.05; SVM: 0.84) and high F1-scores (MEC: 0.89; SVM: 0.92). However the results varied widely depending on the classified data element. Preprocessing methods increased this effect and had significant impact on the classification, both positive and negative. The automatic diagnosis splitter increased the average error rate significantly, even if the F1-score decreased only slightly.

CONCLUSIONS

The low average error rates and high average F1-scores of each pipeline demonstrate the suitability of the investigated NPL methods. However, it was also shown that there is no best practice for an automatic classification of data elements from free-text diagnostic reports.

Building a recruitment database for asthma trials: a conceptual framework for the creation of the UK Database of Asthma Research Volunteers

Background

Randomised clinical trials are the ‘gold standard’ for evaluating the effectiveness of healthcare interventions. However, successful recruitment of participants remains a key challenge for many trialists. In this paper, we present a conceptual framework for creating a digital, population-based database for the recruitment of asthma patients into future asthma trials in the UK. Having set up the database, the goal is to then make it available to support investigators planning asthma clinical trials.

Methods

The UK Database of Asthma Research Volunteers will comprise a web-based front-end that interactively allows participant registration, and a back-end that houses the database containing participants’ key relevant data. The database will be hosted and maintained at a secure server at the Asthma UK Centre for Applied Research based at The University of Edinburgh. Using a range of invitation strategies, key demographic and clinical data will be collected from those pre-consenting to consider participation in clinical trials. These data will, with consent, in due course, be linkable to other healthcare, social, economic, and genetic datasets. To use the database, asthma investigators will send their eligibility criteria for participant recruitment; eligible participants will then be informed about the new trial and asked if they wish to participate. A steering committee will oversee the running of the database, including approval of usage access. Novel communication strategies will be utilised to engage participants who are recruited into the database in order to avoid attrition as a result of waiting time to participation in a suitable trial, and to minimise the risk of their being approached when already enrolled in a trial.

Results

The value of this database will be whether it proves useful and usable to researchers in facilitating recruitment into clinical trials on asthma and whether patient privacy and data security are protected in meeting this aim.

Conclusions

Successful recruitment is fundamental to the success of a clinical trial. The UK Database of Asthma Research Volunteers, the first of its kind in the context of asthma, presents a novel approach to overcoming recruitment barriers and will facilitate the catalysing of important clinical trials on asthma in the UK.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-016-1381-6) contains supplementary material, which is available to authorized users.

Validation of the Hospital Episode Statistics Outpatient Dataset in England

OBJECTIVES

The Hospital Episode Statistics (HES) dataset is a source of administrative 'big data' with potential for costing purposes in economic evaluations alongside clinical trials. This study assesses the validity of coverage in the HES outpatient dataset.

METHODS

Men who died of, or with, prostate cancer were selected from a prostate-cancer screening trial (CAP, Cluster randomised triAl of PSA testing for Prostate cancer). Details of visits that took place after 1/4/2003 to hospital outpatient departments for conditions related to prostate cancer were extracted from medical records (MR); these appointments were sought in the HES outpatient dataset based on date. The matching procedure was repeated for periods before and after 1/4/2008, when the HES outpatient dataset was accredited as a national statistic.

RESULTS

4922 outpatient appointments were extracted from MR for 370 men. 4088 appointments recorded in MR were identified in the HES outpatient dataset (83.1%; 95% confidence interval [CI] 82.0-84.1). For appointments occurring prior to 1/4/2008, 2195/2755 (79.7%; 95% CI 78.2-81.2) matches were observed, while 1893/2167 (87.4%; 95% CI 86.0-88.9) appointments occurring after 1/4/2008 were identified (p for difference <0.001). 215/370 men (58.1%) had at least one appointment in the MR review that was unmatched in HES, 155 men (41.9%) had all their appointments identified, and 20 men (5.4%) had no appointments identified in HES.

CONCLUSIONS

The HES outpatient dataset appears reasonably valid for research, particularly following accreditation. The dataset may be a suitable alternative to collecting MR data from hospital notes within a trial, although caution should be exercised with data collected prior to accreditation.

The effect of sex and age on the comorbidity burden of OSA: an observational analysis from a large nationwide US health claims database

Obstructive sleep apnoea (OSA) is a highly prevalent condition but studies exploring the burden of OSA-associated comorbidities have been limited by small sample sizes with underrepresentation of women.

We queried the Truven Health MarketScan Research Databases 2003–2012, which is a collection of health insurance claims for working adults and retirees with employer-sponsored health insurance. Adults with a diagnostic code for OSA with at least 12 months of follow-up from the index date of OSA diagnosis were compared to a matched random sample. Comorbidities were assessed using International Classification of Diseases, Ninth Edition, codes. A logistic regression model was constructed to test the independent association between OSA and comorbidities.

Our cohort included 1 704 905 patients with OSA and 1 704 417 matched controls. All comorbidities were significantly more prevalent in OSA patients. Type 2 diabetes and ischaemic heart disease were more prevalent in men but hypertension and depression were more prevalent in women with OSA. In contrast, the sex differences in the prevalence of congestive heart failure, arrhythmias and stroke were less pronounced. The prevalence of comorbidities increased with age but the effect of age varied based on the specific comorbidity. The divergence between OSA and controls was more pronounced after the sixth decade of life for most cardiovascular diseases (i.e.heart failure, ischaemic heart disease, stroke and arrhythmias), while depression exhibited an opposite trend. In a fully adjusted model, the odds of all comorbidities were significantly increased in OSA patients.

In a large, nationally representative sample of working and retired people, OSA is strongly associated with significant comorbidities in both men and women with unique sex differences emerging.

Acute Care QUAliTy in chronic Kidney disease (ACQUATIK): a prospective cohort study exploring outcomes of patients with chronic kidney disease

INTRODUCTION

Chronic kidney disease (CKD) is common and carries a high risk of morbidity, including hospital admissions and readmissions and mortality. This is largely attributed to an increased risk of cardiovascular disease. Patients with CKD are less likely to receive evidence-based treatments for cardiovascular disease. However, these treatments are based on trials which generally exclude patients with CKD. It is therefore unclear whether this patient group derives the same benefits without an increased risk of adverse effects.

METHODS AND ANALYSIS

The Acute Care QUAliTy in chronic Kidney disease (ACQUATIK) study is a prospective, observational, multicentre cohort study. Over 4000 patients will be recruited with an enrolment period of 2 years and a follow-up period of 2-4 years. Patients under follow-up by a renal team will be excluded. Data will be obtained from patient and hospital records during the index admission. Preadmission data will be extracted from general practice records based on the Quality and Outcomes Framework. Diagnosis, comorbidities and procedure data pertaining to the index and subsequent admissions will be extracted from the Hospital Episode Statistics database and long-term mortality data will be tracked using the Office of National Statistics. This information will allow us to examine a complete patient journey through primary and secondary care, providing unequalled levels of information on treatment and outcomes of patients with CKD. The combined data set will be used to compare outcomes and treatments among patients with CKD versus patients without CKD. The primary end point is hospital readmission rates. The relationship between age, sex, ethnicity, socioeconomic status and concurrent comorbidities will be analysed to determine their influence on outcomes and treatments.

ETHICS AND DISSEMINATION

The ACQUATIK study has been approved by the NRES Committee West Midlands-South Birmingham-Reference 13/WM/0317. The results from ACQUATIK will be submitted for publication in peer-reviewed journals and presented at primary and secondary care conferences.

TRIAL REGISTRATION NUMBER

ISRCTN37237454.

Agreement between public register and adjudication committee outcome in a cardiovascular randomized clinical trial

The objective of this study is to describe the agreement between randomized trial outcome assessment by committee and outcomes entirely identified through public registers.

METHODS

In the CLARICOR trial, 4,372 patients with stable coronary heart disease received a short course of clarithromycin versus placebo and were followed up for 2.6 years. The pertinent hospital records and death certificates had originally been evaluated by the adjudication committee using common definitions of outcomes mapped into a 6-category list. We now mechanically converted the International Classification of Diseases-coded diagnoses of the public registries into the same categories. After cross-tabulation of the committee diagnoses with National Patient Register diagnoses and Register of Causes of Death, we calculate agreement and compare the estimated intervention effects of the 2 data sets.

RESULTS

With public register data, the protocol-specified categories were slightly more frequent. Overall agreement was 74% for hospital discharges and 60% for cause of death, but the intervention effect, expressed as a hazard ratio, stayed within 4% of the value originally obtained with the adjudication committee (P ≥ .35).

CONCLUSIONS

Our results show a modest agreement between formal adjudication and outcomes deducible from public registers. However, the estimated intervention effect did not differ noticeably between the 2 data sources. If studies on a wide range of public registers confirm these findings, register outcomes may be considered as a replacement for adjudication committees.

The ligurian human immunodeficiency virus clinical network: a web tool to manage patients with human immunodeficiency virus in primary care and multicenter clinical trials

Background

In recent years, Highly-Active Anti-Retroviral Therapies (HAARTs) have modified the Human Immunodeficiency Virus (HIV) life-cycle and the disease is now considered chronic. Consequently, a longitudinal and complex follow-up is now required for HIV positive patients during their lifetime. Moreover, patients often encounter various complications due to comorbidities, related to the immunodeficiency state and HAARTs’ side effects. Thus, HIV positive patients are involved in multicenter clinical trials (MCTs) to improve treatments and discover a preventive vaccine. Therefore, physicians require proper instruments to access comprehensive patient data for managing patients during follow-ups, and tools for data collection and analysis in MCTs.

Objective

The Ligurian HIV Clinical Network aims to provide physicians with a Web-tool to administrate HIV positive patients’ data within primary-care and to reuse the collected clinical information to perform MCTs in Northern Italy.

Methods

The key aspect of the system is a relational database which allows the storage of various types of clinical information (eg, related to HIV, cardiovascular, or hepatic diseases) in multiple formats. The modular design of the database permits a rapid insertion of new parameters without requiring any changes in the database structure. Furthermore, codes from biomedical ontologies controlled vocabularies (“Logical Observation Identifier Names and Codes”, and “International Classification of Diseases 9”) and ontologies (“Systematized Nomenclature of Medicine Clinical Terms”), units and normality ranges used by all partners participating in the project were collected to achieve a complete semantic interoperability. Accordingly, data can be automatically normalized through the z score formula and physicians can extract and correctly compare information with external statistical tools. Moreover, to respect patients’ privacy and legal issues, a local identifier, determined through an HASH cryptography algorithm, is assigned to each patient during the registration process. The database is managed by a user-friendly Web-platform which allows quick access to information during medical examinations and the reusing of the collected data for present and future MCTs. Furthermore, a bidirectional middleware was created in order to import/export information through HL7 messaging. Hence, data can be manually entered by physicians or automatically collected within HL7-compliant Hospital Information systems.

Results

Presently, the direct storage of patients’ information from the San Paolo Hospital (Savona, Italy), and San Martino and Galliera hospitals in Genoa is in a test phase. Currently, 8 centers of Infectious Diseases (located in Liguria and Piedmont) are participating in the project and almost 400 HIV positive patients have been recorded in the system. Patient data has been used for primary care and research purposes. Currently, there are 4 on-going MCTs and preliminary results have already been presented at International HIV congresses.

Conclusions

The Web-platform allows effective management, sharing and reuse of information within primary care and clinical research. In the future it is planned to share the clinical information from this network with other HL7-compliant workgroups and to extend the platform to other infective diseases (eg, hepatitis).

A pragmatic method for electronic medical record-based observational studies: developing an electronic medical records retrieval system for clinical research

OBJECTIVE

The use of electronic medical record (EMR) data is necessary to improve clinical research efficiency. However, it is not easy to identify patients who meet research eligibility criteria and collect the necessary information from EMRs because the data collection process must integrate various techniques, including the development of a data warehouse and translation of eligibility criteria into computable criteria. This research aimed to demonstrate an electronic medical records retrieval system (ERS) and an example of a hospital-based cohort study that identified both patients and exposure with an ERS. We also evaluated the feasibility and usefulness of the method.

DESIGN

The system was developed and evaluated.

PARTICIPANTS

In total, 800 000 cases of clinical information stored in EMRs at our hospital were used.

PRIMARY AND SECONDARY OUTCOME MEASURES

The feasibility and usefulness of the ERS, the method to convert text from eligible criteria to computable criteria, and a confirmation method to increase research data accuracy.

RESULTS

To comprehensively and efficiently collect information from patients participating in clinical research, we developed an ERS. To create the ERS database, we designed a multidimensional data model optimised for patient identification. We also devised practical methods to translate narrative eligibility criteria into computable parameters. We applied the system to an actual hospital-based cohort study performed at our hospital and converted the test results into computable criteria. Based on this information, we identified eligible patients and extracted data necessary for confirmation by our investigators and for statistical analyses with our ERS.

CONCLUSIONS

We propose a pragmatic methodology to identify patients from EMRs who meet clinical research eligibility criteria. Our ERS allowed for the efficient collection of information on the eligibility of a given patient, reduced the labour required from the investigators and improved the reliability of the results.

Assessment of nursing home residents in Europe: the Services and Health for Elderly in Long TERm care (SHELTER) study

BACKGROUND

Aims of the present study are the following: 1. to describe the rationale and methodology of the Services and Health for Elderly in Long TERm care (SHELTER) study, a project funded by the European Union, aimed at implementing the interRAI instrument for Long Term Care Facilities (interRAI LTCF) as a tool to assess and gather uniform information about nursing home (NH) residents across different health systems in European countries; 2. to present the results about the test-retest and inter-rater reliability of the interRAI LTCF instrument translated into the languages of participating countries; 3 to illustrate the characteristics of NH residents at study entry.

METHODS

A 12 months prospective cohort study was conducted in 57 NH in 7 EU countries (Czech Republic, England, Finland, France, Germany, Italy, The Netherlands) and 1 non EU country (Israel). Weighted kappa coefficients were used to evaluate the reliability of interRAI LTCF items.

RESULTS

Mean age of 4156 residents entering the study was 83.4 ± 9.4 years, 73% were female. ADL disability and cognitive impairment was observed in 81.3% and 68.0% of residents, respectively. Clinical complexity of residents was confirmed by a high prevalence of behavioral symptoms (27.5% of residents), falls (18.6%), pressure ulcers (10.4%), pain (36.0%) and urinary incontinence (73.5%). Overall, 197 of the 198 the items tested met or exceeded standard cut-offs for acceptable test-retest and inter-rater reliability after translation into the target languages.

CONCLUSION

The interRAI LTCF appears to be a reliable instrument. It enables the creation of databases that can be used to govern the provision of long-term care across different health systems in Europe, to answer relevant research and policy questions and to compare characteristics of NH residents across countries, languages and cultures.

Integrating clinical research with the Healthcare Enterprise: from the RE-USE project to the EHR4CR platform

BACKGROUND

There are different approaches for repurposing clinical data collected in the Electronic Healthcare Record (EHR) for use in clinical research. Semantic integration of "siloed" applications across domain boundaries is the raison d'être of the standards-based profiles developed by the Integrating the Healthcare Enterprise (IHE) initiative - an initiative by healthcare professionals and industry promoting the coordinated use of established standards such as DICOM and HL7 to address specific clinical needs in support of optimal patient care. In particular, the combination of two IHE profiles - the integration profile "Retrieve Form for Data Capture" (RFD), and the IHE content profile "Clinical Research Document" (CRD) - offers a straightforward approach to repurposing EHR data by enabling the pre-population of the case report forms (eCRF) used for clinical research data capture by Clinical Data Management Systems (CDMS) with previously collected EHR data.

OBJECTIVE

Implement an alternative solution of the RFD-CRD integration profile centered around two approaches: (i) Use of the EHR as the single-source data-entry and persistence point in order to ensure that all the clinical data for a given patient could be found in a single source irrespective of the data collection context, i.e. patient care or clinical research; and (ii) Maximize the automatic pre-population process through the use of a semantic interoperability services that identify duplicate or semantically-equivalent eCRF/EHR data elements as they were collected in the EHR context.

METHODS

The RE-USE architecture and associated profiles are focused on defining a set of scalable, standards-based, IHE-compliant profiles that can enable single-source data collection/entry and cross-system data reuse through semantic integration. Specifically, data reuse is realized through the semantic mapping of data collection fields in electronic Case Report Forms (eCRFs) to data elements previously defined as part of patient care-centric templates in the EHR context. The approach was evaluated in the context of a multi-center clinical trial conducted in a large, multi-disciplinary hospital with an installed EHR.

RESULTS

Data elements of seven eCRFs used in a multi-center clinical trial were mapped to data elements of patient care-centric templates in use in the EHR at the George Pompidou hospital. 13.4% of the data elements of the eCRFs were found to be represented in EHR templates and were therefore candidate for pre-population. During the execution phase of the clinical study, the semantic mapping architecture enabled data persisted in the EHR context as part of clinical care to be used to pre-populate eCRFS for use without secondary data entry. To ensure that the pre-populated data is viable for use in the clinical research context, all pre-populated eCRF data needs to be first approved by a trial investigator prior to being persisted in a research data store within a CDMS.

CONCLUSION

Single-source data entry in the clinical care context for use in the clinical research context - a process enabled through the use of the EHR as single point of data entry, can - if demonstrated to be a viable strategy - not only significantly reduce data collection efforts while simultaneously increasing data collection accuracy secondary to elimination of transcription or double-entry errors between the two contexts but also ensure that all the clinical data for a given patient, irrespective of the data collection context, are available in the EHR for decision support and treatment planning. The RE-USE approach used mapping algorithms to identify semantic coherence between clinical care and clinical research data elements and pre-populate eCRFs. The RE-USE project utilized SNOMED International v.3.5 as its "pivot reference terminology" to support EHR-to-eCRF mapping, a decision that likely enhanced the "recall" of the mapping algorithms. The RE-USE results demonstrate the difficult challenges involved in semantic integration between the clinical care and clinical research contexts.

Routine data from hospital information systems can support patient recruitment for clinical studies

BACKGROUND

Delayed patient recruitment is a common problem in clinical studies. Hospital information systems (HIS) contain data items relevant for inclusion or exclusion criteria of these studies.

PURPOSE

We developed and assessed a system to support patient recruitment using HIS data.

METHODS

We developed a workflow integrated in our HIS to notify study physicians about potential trial subjects. Automatic HIS database queries based on inclusion and exclusion criteria for each clinical study are performed regularly and generate e-mail notifications via a communication server. Study physicians can verify eligibility with a specific HIS study module. The system performance was assessed with a survey addressing utility, usability, stability, change in recruitment rate, and estimated time savings.

RESULTS

During 10 months of operation, 1328 notifications were generated and 329 enrollments (25%) were documented for seven studies. Precision of alerts depends on availability of appropriate HIS items. Utility and usability were assessed as good, and stability as excellent. Users reported an increased patient recruitment rate for three studies. Three studies reported an estimated time saving of 10 min per recruited patient. The main perceived benefit was systematic identification of potentially eligible patients without time-consuming patient screening procedures in the different parts of the hospital.

LIMITATIONS

Notifications about potentially eligible patients depend on HIS data quality regarding inclusion/exclusion criteria, in particular, completeness, timeliness, and validity.

CONCLUSIONS

Routine HIS data can support patient recruitment for clinical studies by means of an automated notification workflow and efficient access to clinical data.

Cross-sectional analysis of a collaborative Web-based database for lupus erythematosus-associated skin lesions: prospective enrollment of 114 patients

OBJECTIVES

To assess disease severity in subsets of patients with cutaneous lupus erythematosus (CLE) by using outcome and quality-of-life measures, and to determine treatment responsiveness by establishing a Web-based database of patients with skin manifestations of lupus.

DESIGN

Prospective, cross-sectional study.

SETTING

University hospital cutaneous autoimmunity outpatient clinic.

PATIENTS

One hundred fourteen patients who presented from January 15, 2007, to November 8, 2007, and met the criteria for having CLE or lupus-nonspecific skin disease.

MAIN OUTCOME MEASURES

Scores on the CLE Disease Activity and Severity Index and the modified Skindex-29 (a quality-of-life measure) completed at each visit.

RESULTS

Seven patients (6.1%) presented with acute CLE, 21 (18.4%) with subacute CLE, 77 (67.5%) with chronic CLE, 7 (6.1%) with systemic lupus erythematosus and LE-nonspecific skin lesions, and 1 (0.9%) with LE-nonspecific skin disease only. The mean baseline CLE Disease Activity and Severity Index activity/damage scores in patients with acute, subacute, and chronic CLE were 6.4/5.1, 11.1/1.6, and 7.5/10.2, respectively. The mean baseline modified Skindex-29 scores were 76.3, 79.4, and 82.7, respectively (P = .80). The disease in 11 of the patients (9.6%) was considered refractory to conventional therapies. Significantly more patients in the refractory group than the nonrefractory group were current smokers (P = .006).

CONCLUSION

This Web-based database should allow collection of data related to disease activity, quality of life, and response to therapy at multiple centers.

Special features of health services and register based trials - experiences from a randomized trial of childbirth classes

BACKGROUND

Evaluating complex interventions in health services faces various difficulties, such as making practice changes and costs. Ways to increase research capacity and decrease costs include making research an integral part of health services and using routine data to judge outcomes. The purpose of this article is to report the feasibility of a pilot trial relying solely on routinely collected register data and being based on ordinary health services.

METHODS

The example intervention was education to public health nurses (PHN) (childbirth classes) to reduce caesarean section rates via pre-delivery considerations of pregnant women. 20 maternity health centers (MHC) were paired and of each 10 pairs, one MHC was randomly allocated to an intervention group and the other to a control; 8 pairs with successful intervention were used in the analyses (1601 mothers). The women visiting to the study maternity centers were identified from the Customer Register of Helsinki City. A list of the study women was made using the mother's personal identification number, visit date, the maternity center code, birth date and gestation length. The mode of delivery and health outcomes were retrieved from the Finnish Medical Birth Register (MBR). Process data of the intervention are based on observations, written feedback and questionnaires from PHNs, and project correspondence.

RESULTS

It took almost two years to establish how to obtain permissions and to actually obtain it for the trial. Obtaining permissions for the customer and outcome data and register linkages was unproblematic and the cluster randomization provided comparable groups. The intervention did not succeed well. Had the main aim of the trial been to cause a change in PHNs behavior, we would have very likely intensified the intervention during the trial.

CONCLUSION

Our experiences encourage the use of trials that obtain their outcomes from registers. Changing the behavior of ordinary health service providers is a challenging intervention.

TRIAL REGISTRATION NUMBER

not registered (see Results).

An examination of the levels of confidence that consultants have in routinely collected clinical information

Anecdotal evidence suggests that consultants have a low level of confidence in routinely collected clinical data. The aims of this study were to assess the actual levels of confidence in clinical information experienced by consultants and to understand the factors affecting confidence. A questionnaire was devised incorporating a scale to measure levels of confidence. This was disseminated to 506 consultants and achieved a 57 per cent response rate. The confidence scale showed good internal reliability and the influential factors were modelled using multilinear regression. Consultant specialty, increased involvement in clinical coding, and participation in service planning/monitoring all had a significant effect. A small number of semi-structured interviews further examined these factors. Whilst the findings confirm there is scope to increase levels of confidence, the mean score indicated neither a high nor a low level of confidence. The results demonstrate that improving consultants' confidence in information is complex and requires a partnership approach.

Optimising workflow in andrology: a new electronic patient record and database

AIM

To improve workflow and usability by introduction of a new electronic patient record (EPR) and database.

METHODS

Establishment of an EPR based on open source technology (MySQL database and PHP scripting language) in a tertiary care andrology center at a university clinic. Workflow analysis, a benchmark comparing the two systems and a survey for usability and ergonomics were carried out.

RESULTS

Workflow optimizations (electronic ordering of laboratory analysis, elimination of transcription steps and automated referral letters) and the decrease in time required for data entry per patient to 71%+/-27%, P<0.05, lead to a workload reduction. The benchmark showed a significant performance increase (highest with starting the respective system: 1.3+/-0.2 s vs. 11.1+/-0.2 s, mean+/-SD). In the survey, users rated the new system at least two ranks higher over its predecessor (P<0.01) in all sub-areas.

CONCLUSION

With further improvements, today's EPR can evolve to substitute paper records, saving time (and possibly costs), supporting user satisfaction and expanding the basis for scientific evaluation when more data is electronically available. Newly introduced systems should be versatile, adaptable for users, and workflow-oriented to yield the highest benefit. If ready-made software is purchased, customization should be implemented during rollout.

Results from the Estonian postmenopausal hormone therapy trial [ISRCTN35338757]

OBJECTIVES

At present the Women's Health Initiative trial is the only reported randomised controlled trial studying the effects of hormone therapy among healthy postmenopausal women. The Women's Health Initiative reports have been criticized for lacking in generalisability, due to the characteristics of the trial population. We aimed to compare the health effects of oral continuous combined hormone therapy with a placebo and non-treatment among healthy Estonian women.

METHODS

Eligible women were randomised into a blind group of hormone therapy versus placebo and into a non-blind group of open label hormone therapy versus non-treatment. One thousand seven hundred and seventy-eight postmenopausal women aged 50-64 at the time of sampling were recruited in 1999-2001 at three clinical centers in Estonia. Participants received conjugated equine oestrogens, 0.625 mg/d, plus medroxyprogesterone acetate, 2.5mg/d, or conjugated equine oestrogens, 0.625 mg/d, plus medroxyprogesterone acetate, 5mg/d, if less than 3 years had passed since menopause at recruitment, or matched placebo or non-treatment. Trial treatment was stopped gradually from 1 January 2004 to 31 May 2004.

RESULTS

After a follow-up period from 2.0 to 5.0 years the combined hazard ratio, stratified by blinding and adjusted for age at recruitment and former oral contraceptive use was 1.12 (95% confidence interval [CI]: 0.90-1.40) for coronary heart disease, 1.24 (95% CI: 0.85-1.82) for cerebrovascular disease, 1.36 (95% CI: 0.73-2.52) for total cancer, and 0.61 (95% CI: 0.42 to 0.89) for bone fractures.

CONCLUSIONS

The results from the Estonian Postmenopausal Hormone Therapy randomised trial are consistent with the Women's Health Initiative findings.

Can electronic routine data act as a surrogate for patient-assessed outcome measures?

Objectives:There has been a rapid growth in the use of patient-assessed outcomes (PAOs) that are measured in the assessment of health technologies. The process of collection of such measures can be costly, and there may be problems associated with the ability of the patient to complete them. The use of electronically stored routine data may reduce costs and overcome the problems associated with patient completion. The feasibility of using routine data surrogates for the UK Inflammatory Bowel Disease Questionnaire (UKIBDQ) and the Short Form 36 (SF-36) was examined.

Methods:Clinical terms and codes for the UKIBDQ and SF-36 questions were identified, and data from electronic routine sources were sought on patients participating in a randomized controlled trial. The presence or absence of relevant symptoms was used to generate surrogate scores, which were compared with the original scores.

Results:Most questions in the UKIBDQ and SF-36 were codable but only one third of the terms were recorded routinely in electronic form. The surrogate total IBDQ score had reasonable reliability (Kuder–Richardson coefficient = 0.51), but this reliability could not be determined for the SF-36. Intraclass correlations between routine and designed data were poor to weak.

Conclusions:Although electronic routine data sources had the capacity to develop surrogate measures for patient assessed outcomes, there was evidence of wide underutilization of coding systems leading to an underreporting of symptoms. This finding is consistent with previous literature where only poor correlations were illustrated between patient assessed outcomes and surrogate scoring of symptoms.

Cross sectional survey of multicentre clinical databases in the United Kingdom

OBJECTIVES

To describe the multicentre clinical databases that exist in the United Kingdom, to report on their quality, to explore which organisational and managerial features are associated with high quality, and to make recommendations for improvements.

DESIGN

Cross sectional survey, with interviews with database custodians and search of electronic bibliographic database (PubMed).

STUDIES REVIEWED

105 clinical databases across the United Kingdom.

RESULTS

Clinical databases existed in all areas of health care, but their distribution was uneven-cancer and surgery were better covered than mental health and obstetrics. They varied greatly in age, size, growth rate, and geographical areas covered. Their scope (and thus their potential uses) and the quality of the data collected also varied. The latter was not associated with any organisational characteristics. Despite impressive achievements, many faced substantial financial uncertainty. Considerable scope existed for improvements: greater use of nationally approved codes; more support from relevant professional organisations; greater involvement by nurses, allied health professionals, managers, and laypeople in database management teams; and more attention to data security and ensuring patient confidentiality. With some notable exceptions, the audit and research potential of most databases had not been realised: half the databases had each produced only four or fewer peer reviewed research articles.

CONCLUSIONS

At least one clinical database support unit is needed in the United Kingdom to provide assistance in organisation and management, information technology, epidemiology, and statistics. Without such an initiative, the variable picture of databases reported here is likely to persist and their potential not be realised.