Are outcome data regarding the survivors of neonatal care available from routine sources?

Developing routinely recorded clinical data from electronic patient records as a national resource to improve neonatal health care: the Medicines for Neonates research programme

Background

Clinical data offer the potential to advance patient care. Neonatal specialised care is a high-cost NHS service received by approximately 80,000 newborn infants each year.

Objectives

(1) To develop the use of routinely recorded operational clinical data from electronic patient records (EPRs), secure national coverage, evaluate and improve the quality of clinical data, and develop their use as a national resource to improve neonatal health care and outcomes. To test the hypotheses that (2) clinical and research data are of comparable quality, (3) routine NHS clinical assessment at the age of 2 years reliably identifies children with neurodevelopmental impairment and (4) trial-based economic evaluations of neonatal interventions can be reliably conducted using clinical data. (5) To test methods to link NHS data sets and (6) to evaluate parent views of personal data in research.

Design

Six inter-related workstreams; quarterly extractions of predefined data from neonatal EPRs; and approvals from the National Research Ethics Service, Health Research Authority Confidentiality Advisory Group, Caldicott Guardians and lead neonatal clinicians of participating NHS trusts.

Setting

NHS neonatal units.

Participants

Neonatal clinical teams; parents of babies admitted to NHS neonatal units.

Interventions

In workstream 3, we employed the Bayley-III scales to evaluate neurodevelopmental status and the Quantitative Checklist of Autism in Toddlers (Q-CHAT) to evaluate social communication skills. In workstream 6, we recruited parents with previous experience of a child in neonatal care to assist in the design of a questionnaire directed at the parents of infants admitted to neonatal units.

Data sources

Data were extracted from the EPR of admissions to NHS neonatal units.

Main outcome measures

We created a National Neonatal Research Database (NNRD) containing a defined extract from real-time, point-of-care, clinician-entered EPRs from all NHS neonatal units in England, Wales and Scotland (n = 200), established a UK Neonatal Collaborative of all NHS trusts providing neonatal specialised care, and created a new NHS information standard: the Neonatal Data Set (ISB 1595) (see http://webarchive.nationalarchives.gov.uk/±/http://www.isb.nhs.uk/documents/isb-1595/amd-32–2012/index_html; accessed 25 June 2018).

Results

We found low discordance between clinical (NNRD) and research data for most important infant and maternal characteristics, and higher prevalence of clinical outcomes. Compared with research assessments, NHS clinical assessment at the age of 2 years has lower sensitivity but higher specificity for identifying children with neurodevelopmental impairment. Completeness and quality are higher for clinical than for administrative NHS data; linkage is feasible and substantially enhances data quality and scope. The majority of hospital resource inputs for economic evaluations of neonatal interventions can be extracted reliably from the NNRD. In general, there is strong parent support for sharing routine clinical data for research purposes.

Limitations

We were only able to include data from all English neonatal units from 2012 onwards and conduct only limited cross validation of NNRD data directly against data in paper case notes. We were unable to conduct qualitative analyses of parent perspectives. We were also only able to assess the utility of trial-based economic evaluations of neonatal interventions using a single trial. We suggest that results should be validated against other trials.

Conclusions

We show that it is possible to obtain research-standard data from neonatal EPRs, and achieve complete population coverage, but we highlight the importance of implementing systematic examination of NHS data quality and completeness and testing methods to improve these measures. Currently available EPR data do not enable ascertainment of neurodevelopmental outcomes reliably in very preterm infants. Measures to maintain high quality and completeness of clinical and administrative data are important health service goals. As parent support for sharing clinical data for research is underpinned by strong altruistic motivation, improving wider public understanding of benefits may enhance informed decision-making.

Future work

We aim to implement a new paradigm for newborn health care in which continuous incremental improvement is achieved efficiently and cost-effectively by close integration of evidence generation with clinical care through the use of high-quality EPR data. In future work, we aim to automate completeness and quality checks and make recording processes more ‘user friendly’ and constructed in ways that minimise the likelihood of missing or erroneous entries. The development of criteria that provide assurance that data conform to prespecified completeness and quality criteria would be an important development. The benefits of EPR data might be extended by testing their use in large pragmatic clinical trials. It would also be of value to develop methods to quality assure EPR data including involving parents, and link the NNRD to other health, social care and educational data sets to facilitate the acquisition of lifelong outcomes across multiple domains.

Study registration

This study is registered as PROSPERO CRD42015017439 (workstream 1) and PROSPERO CRD42012002168 (workstream 3).

Funding

The National Institute for Health Research Programme Grants for Applied Research programme (£1,641,471). Unrestricted donations were supplied by Abbott Laboratories (Maidenhead, UK: £35,000), Nutricia Research Foundation (Schiphol, the Netherlands: £15,000), GE Healthcare (Amersham, UK: £1000). A grant to support the use of routinely collected, standardised, electronic clinical data for audit, management and multidisciplinary feedback in neonatal medicine was received from the Department of Health and Social Care (£135,494).

Validity of neurodevelopmental outcomes of children born very preterm assessed during routine clinical follow-up in England

OBJECTIVE

To determine the validity of assessing and recording the neurodevelopmental outcome of very preterm infants during routine clinical follow-up in England.

DESIGN

Children born <30 weeks gestation, attending routine clinical follow-up at post-term ages 20-28 months, were recruited. Data on neurodevelopmental outcomes were recorded by the reviewing clinician in a standardised format in the child's electronic patient record, based on a set of key questions designed to be used without formal training or developmental testing. Using a predefined algorithm, each participant was classified as having 'no', 'mild/moderate' or 'severe' impairment in cognitive, communication and motor domains. All participants also received a research assessment by a single assessor using the Bayley Scales of Infant Development, third edition (Bayley-III). The sensitivity and specificity of routine data in capturing impairment (any Bayley-III score <85) or severe impairment (any Bayley-III score <70) was calculated.

RESULTS

190 children participated. The validity of routine assessments in identifying children with no impairment and no severe impairment was high across all domains (specificities 83.9%-100.0% and 96.6%-100.0%, respectively). However, identification of impairments, particularly in the cognitive (sensitivity 69.7% (55.1%-84.3%)) and communication (sensitivity (53.2% (42.0%-64.5%)) domains, was poor.

CONCLUSIONS

Neurodevelopmental status determined during routine clinical assessment lacks adequate sensitivity in cognitive and communication domains. It is uncertain whether this reflects the assessment or/and the recording of findings. As early intervention may improve education and social outcomes, this is an important area for healthcare quality improvement research.

Evaluation of the use of a parent questionnaire to provide later health status data: the PANDA study

BACKGROUND

Routine comparable outcome data collection relating to the later health status of babies born very preterm has long been considered important, but has not been achieved in the UK.

AIM

To test the potential for a parental questionnaire to provide these data for all eligible babies from a geographical population.

METHODS

Consent for follow-up by questionnaire (using the Parent Report of Children's Abilities-Revised combined with questions derived from the Oxford minimum dataset) was sought for all babies ≤30 weeks of gestation, discharged from a hospital in the East Midlands and Yorkshire regions of the UK, having been born between 1 January 2007 and 31 December 2011.

RESULTS

The rate of consent to participate in follow-up showed a steady increase over time to 83.1% in 2011. However, the response rate in terms of completion and return of the questionnaire at 2 years, as a proportion of those eligible, showed little change over time, varying between 42% and 46%. Among those children where a questionnaire was returned, the rate of disability was broadly consistent over time: lowest in 2009, 21.0% (95% CI 16.8% to 25.6%) and highest in 2011, 25.5% (95% CI 21.5% to 31.2%). The instruments used appeared effective with the capability of discriminating between children with physical and/or cognitive disability.

CONCLUSIONS

The overall response rate in terms of returned questionnaires was disappointing and inadequate to recommend for implementation. It is possible that response rates would have been higher had clinical follow-up been linked to the data obtained from the questionnaires rather than running as a parallel process.

Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research

OBJECTIVE

To review the methods and dimensions of data quality assessment in the context of electronic health record (EHR) data reuse for research.

MATERIALS AND METHODS

A review of the clinical research literature discussing data quality assessment methodology for EHR data was performed. Using an iterative process, the aspects of data quality being measured were abstracted and categorized, as well as the methods of assessment used.

RESULTS

Five dimensions of data quality were identified, which are completeness, correctness, concordance, plausibility, and currency, and seven broad categories of data quality assessment methods: comparison with gold standards, data element agreement, data source agreement, distribution comparison, validity checks, log review, and element presence.

DISCUSSION

Examination of the methods by which clinical researchers have investigated the quality and suitability of EHR data for research shows that there are fundamental features of data quality, which may be difficult to measure, as well as proxy dimensions. Researchers interested in the reuse of EHR data for clinical research are recommended to consider the adoption of a consistent taxonomy of EHR data quality, to remain aware of the task-dependence of data quality, to integrate work on data quality assessment from other fields, and to adopt systematic, empirically driven, statistically based methods of data quality assessment.

CONCLUSION

There is currently little consistency or potential generalizability in the methods used to assess EHR data quality. If the reuse of EHR data for clinical research is to become accepted, researchers should adopt validated, systematic methods of EHR data quality assessment.

How should we report neonatal outcomes?

The aim of neonatal care is to achieve long-term survival free of handicap. There is a need for standardised datasets linking perinatal care to outcome at 2 years corrected age for all babies who have gone through neonatal intensive care. Realistically this can only happen if all the data are collected by the units caring for the babies. This has not been possible previously using routinely collected data because of the poor quality of such information. Recent improvements in neonatal data collection along with the development of standardised neonatal and follow-up datasets make it possible that this could now be achieved from routine data collected as part of everyday clinical care. It is important that further links with maternity and child health systems are developed. The National Neonatal Audit Project, funded by the Department of Health, will hopefully develop the infrastructure to allow the storage, analysis and rapid reporting of pooled neonatal and follow-up data.

A Framework for the Development of MaternalQuality of Care Indicators

BACKGROUND

In collaboration with the California Department of Health Maternal and Child Health Branch, the authors formed a Working Group to identify potential clinical indicators that could be used to inform decision making regarding maternal health care quality.

OBJECTIVE

To develop potential indicators for the assessment of maternal health care quality.

MATERIALS AND METHODS

A Working Group was convened to review information from the published literature and expert opinion. Selection of potential indicators was guided by the following goals: 1) To identify key areas for routine aggregate monitoring; 2) To include perspectives of relevant stakeholders in maternal health care services; 3) To include measures that are comprehensive and reflect a balance between maternal and fetal interests; and 4) To develop measures that would be valid, generalizable, mutable, and feasible.

RESULTS

Ninety potential indicators were identified. Each underwent a thorough review based on: its definition, objective, and validity; its contribution to innovation; the cost and timeliness of implementation; its feasibility, acceptability, and potential effectiveness; and its compatibility with ethics, values, and social policy. This process yielded 24 final indicators from the following categories: Health Status and Access (e.g., availability of 24 h inpatient anesthesia); Preconception and Interconception Care (e.g., Pap smear use); Antenatal Care (e.g., hospitalization for uncontrolled diabetes or pyelonephritis); Labor and Delivery Care (e.g., chorioamnionitis or obstetrical hemorrhage), and Postpartum Care (e.g., rate of postpartum visits).

CONCLUSIONS

These potential indicators, representative of the women's health continuum, can serve as a foundation to structure the development of consensus and methods for maternal health care quality assessment.

Quality of life of 6-year-old survivors from a regional neonatal unit

OBJECTIVE

Quality of life (QOL) of children who required treatment in a national neonatal intensive care unit (NICU) during the first week of life was determined at 6 years of age.

METHOD

QOL was assessed using a standardized questionnaire employing a multi-attribute scoring system including the functions: hearing, vision, speech, mobility, emotion, learning, self-care and pain. After excluding those with congenital neurodevelopmental disorders, questionnaire returns were analyzed from 177 children treated in the NICU in 1990 and a comparative age-matched group of 230 children who did not require neonatal care. Of these, returns were obtained from 143 (81%) cases and 171 (74%) of the non-treated group.

RESULTS

There was no difference in overall ability between the two groups, with 95 (66%) of cases and 126 (74%) of the comparison group reporting normal scores in all functions. Children treated in the NICU had decreased scores in individual functions including speech (p = 0.04), mobility (p = 0.009) and self-care (p = 0.006). For the study population, males had lower function in speech (p = 0.04) and learning (p = 0.001), with significantly worse function overall (p = 0.02) when compared with female cases. When compared with same-gender children who did not require NICU care, overall function was also significantly worse for male but not female cases (p = 0.0002), and this was largely contributed to by impairment in speech (p = 0.03), mobility (p = 0.04), learning abilities (p = 0.02) and self-care (p = 0.03). Eleven (7.7%) cases compared with just two (1.2%) children who were not treated in the NICU required assistance at school (p = 0.009). No difference was observed when QOL was assessed according to gestational age and birth weight.

CONCLUSION

Using a simple scoring system this study has shown that, for survivors who required early neonatal intensive care, the QOL at 6 years compared favorably with that of children not treated in the NICU, especially for girls.

Measuring later health status of high risk infants: randomised comparison of two simple methods of data collection

OBJECTIVE

To test two methods of providing low cost information on the later health status of survivors of neonatal intensive care.

DESIGN

Cluster randomised comparison.

SETTING

Nine hospitals distributed across two UK health regions. Each hospital was randomised to use one of two methods of follow up.

PARTICIPANTS

All infants born

METHOD

Families were recruited at the time of discharge. In one method of follow up families were asked to complete a questionnaire about their child's health at the age of 2 years (corrected for gestation). In the other method the children's progress was followed by clerks in the local community child health department by using sources of routine information.

RESULTS

236 infants were recruited to each method of follow up. Questionnaires were returned by 214 parents (91%; 95% confidence interval 84% to 97%) and 223 clerks (95%; 86% to 100%). Completed questionnaires were returned by 201 parents (85%; 76% to 94%) and 158 clerks (67%; 43% to 91%). Most parents found the forms easy to complete, but some had trouble understanding the concept of "corrected age" and hence when to return the form. Community clerks often had to rely on information that was out of date and difficult to interpret.

CONCLUSION

Neither questionnaires from parents nor routinely collected health data are adequate methods of providing complete follow up data on children who were born preterm and required neonatal intensive care, though both methods show potential.

Collapse

Key Words Collapse

MESH Headings

• Community Health Services/methods
• Costs and Cost Analysis
• Data Collection/economics
• Data Collection/methods
• Follow-Up Studies
• Health Status Indicators
• Humans
• Infant, Newborn
• Infant, Premature
• Intensive Care, Neonatal
• Parents
• Prognosis
• Reproducibility of Results
• Surveys and Questionnaires

Collapse

Grants Collapse

Affiliation(s)

D Field University of Leicester Medical School, Leicester, UK.
E S Draper
M J Gompels
C Green
A Johnson
D Shortland
M Blair
B Manktelow
C R Lamming
C Law

Collapse

Risk adjusted and population based studies of the outcome for high risk infants in Scotland and Australia. International Neonatal Network, Scottish Neonatal Consultants, Nurses Collaborative Study Group

OBJECTIVES

To compare outcomes of care in selected neonatal intensive care units (NICUs) for very low birthweight (VLBW) or preterm infants in Scotland and Australia (study 1) and perinatal care for all VLBW infants in both countries (study 2).

DESIGN

Study 1: risk adjusted cohort study; study 2: population based cohort study.

SUBJECTS

Study 1: all 2621 infants of < 1500 g birth weight or < 31 weeks' gestation admitted to a volunteer sample of hospitals comprising eight of all 17 Scottish NICUs and six of all 12 tertiary NICUs in New South Wales and Queensland in 1993-1994; study 2: all 5986 infants of 500-1499 g birth weight registered as live born in Scotland and Australia in 1993-1994.

MAIN OUTCOMES

Study 1: (a) hospital death; (b) death or cerebral damage, each adjusted for gestation and CRIB (clinical risk index for babies); study 2: neonatal (28 day) mortality.

RESULTS

Study 1. Data were obtained for 1628 admissions in six Australian NICUs, 775 in five Scottish tertiary NICUs, and 148 in three Scottish non-tertiary NICUs. Crude hospital death rates were 13%, 22%, and 22% respectively. Risk adjusted hospital mortality was about 50% higher in Scottish than in Australian NICUs (adjusted mortality ratio 1.46, 95% confidence interval (CI) 1.29 to 1.63, p < 0.001). There was no difference in risk adjusted outcomes between Scottish tertiary and non-tertiary NICUs. After risk adjustment, death or cerebral damage was more common in Scottish than Australian NICUs (odds ratio 1.9, 95% CI 1.5 to 2.5). Both these risk adjusted adverse outcomes remained more common in Scottish than Australian NICUs after excluding all infants < 28 weeks' gestation from the comparison. Study 2. Population based neonatal mortality in infants of 500-1499 g was higher in Scotland (20.3%) than Australia (16.6%) (relative risk 1.22, 95% CI 1.08 to 1. 39, p = 0.002). In a post hoc analysis, neonatal mortality was also higher in England and Wales than in Australia.

CONCLUSIONS

Study 1: outcome was better in the Australian NICUs. Study 2: perinatal outcome was better in Australia. Both results may be consistent, at least in part, with differences in the organisation and implementation of neonatal care.

Health status of a population of infants born before 26 weeks gestation derived from routine data collected between 21 and 27 months post-delivery

This retrospective study was designed: (a) to determine the extent to which routine data sources in the UK can provide data relating to the later health status of selected groups of infants; and (b) to use such an approach to describe the outcome of a geographically defined population of infants born before 26 weeks gestation. All infants of less than 26 weeks gestation admitted for neonatal intensive care during the period 1/1/91 and 31/12/93 whose mother's address at the time of birth was within the boundaries of the Trent Health Region were included. Health status was assessed against a previously described simple scheme and using information from existing sources only. During the 3-year period 249 infants of less than 26 weeks gestation were admitted for intensive care. Of these 66 (26.5%) survived to be discharged from the neonatal service. A further seven infants died before the age of 2 years. Of the remaining 59 four were lost to follow up (three could not be traced; one was living abroad). Of the 55 infants reviewed, 36 demonstrated no features, pre-defined in the classification scheme, of severe disability. However, only 30 children appeared to be considered entirely normal.

CONCLUSION

Infants born before 26 weeks gestation and admitted for neonatal intensive care had, approximately, a 12% chance of normal survival to 2 years. A slightly smaller proportion of infants survived with significant disability. Existing routine data sources could be adapted to provide useful public health information about the outcome of 'high risk' groups of infants.