Robust metrics for assessing the performance of different verbal autopsy cause assignment methods in validation studies

R-index: a standardized representativeness metric for benchmarking diversity, equity, and inclusion in biopharmaceutical clinical trial development

Background

Diversity, equity, and inclusion pertaining to race, ethnicity, and related concepts have historically been underrepresented in clinical trials for pharmaceutical drug development, although this is an increasing topic for regulators, payers, and patient advocacy groups. We aimed to develop a summary statistical measure to assess such representativeness.

Methods

A statistical measure using population demographic parameters derived from performance metrics through verbal autopsy research was proposed for using population frameworks in the UK. The summary measure, R-index, was demonstrated using simulation data with population frameworks from the UK (116 Roche UK clinical trials 2013-2022) and then using published clinical trial results (NCT02366143 [March 1, 2015-September 15, 2017], NCT04368728 [July 27, 2020-October 9, 2020], and NCT04470427 [July 27, 2020-November 25, 2020]). R-index was further proposed for use with benchmarking performance in representative trial development for internal processes, external benchmarking, and performance tracking in clinical trial development.

Findings

R-index was derived from a standardized statistical measure called the L1 norm, or Manhattan distance, and then normalized to the maximum theoretical error observed in some populations using population framework or ontology for reporting concepts such as race, ethnicity, and other dimensions of diversity used to characterize patient cohorts. R-index demonstrated desirable qualities in demonstration simulations, including a range of 0-1, ease of calculation and use, and interpretability and flexibility, as data standards in the space of inclusive research continue to develop.

Interpretation

R-index is an interpretable, accessible summary statistic that may be useful for tracking and benchmarking representativeness in inclusive research and related domains. R-index is adaptable to different population frameworks and ontologies across different settings and considerations in terms of underlying population variables.

Funding

F. Hoffmann-La Roche Ltd/Genentech, Inc.

Tree-informed Bayesian multi-source domain adaptation: cross-population probabilistic cause-of-death assignment using verbal autopsy

Determining causes of deaths (CODs) occurred outside of civil registration and vital statistics systems is challenging. A technique called verbal autopsy (VA) is widely adopted to gather information on deaths in practice. A VA consists of interviewing relatives of a deceased person about symptoms of the deceased in the period leading to the death, often resulting in multivariate binary responses. While statistical methods have been devised for estimating the cause-specific mortality fractions (CSMFs) for a study population, continued expansion of VA to new populations (or "domains") necessitates approaches that recognize between-domain differences while capitalizing on potential similarities. In this article, we propose such a domain-adaptive method that integrates external between-domain similarity information encoded by a prespecified rooted weighted tree. Given a cause, we use latent class models to characterize the conditional distributions of the responses that may vary by domain. We specify a logistic stick-breaking Gaussian diffusion process prior along the tree for class mixing weights with node-specific spike-and-slab priors to pool information between the domains in a data-driven way. The posterior inference is conducted via a scalable variational Bayes algorithm. Simulation studies show that the domain adaptation enabled by the proposed method improves CSMF estimation and individual COD assignment. We also illustrate and evaluate the method using a validation dataset. The article concludes with a discussion of limitations and future directions.

BAYESIAN NESTED LATENT CLASS MODELS FOR CAUSE-OF-DEATH ASSIGNMENT USING VERBAL AUTOPSIES ACROSS MULTIPLE DOMAINS

Understanding cause-specific mortality rates is crucial for monitoring population health and designing public health interventions. Worldwide, two-thirds of deaths do not have a cause assigned. Verbal autopsy (VA) is a well-established tool to collect information describing deaths outside of hospitals by conducting surveys to caregivers of a deceased person. It is routinely implemented in many low- and middle-income countries. Statistical algorithms to assign cause of death using VAs are typically vulnerable to the distribution shift between the data used to train the model and the target population. This presents a major challenge for analyzing VAs, as labeled data are usually unavailable in the target population. This article proposes a latent class model framework for VA data (LCVA) that jointly models VAs collected over multiple heterogeneous domains, assigns causes of death for out-of-domain observations and estimates cause-specific mortality fractions for a new domain. We introduce a parsimonious representation of the joint distribution of the collected symptoms using nested latent class models and develop a computationally efficient algorithm for posterior inference. We demonstrate that LCVA outperforms existing methods in predictive performance and scalability. Supplementary Material and reproducible analysis codes are available online. The R package LCVA implementing the method is available on GitHub (https://github.com/richardli/LCVA).

Agreement between cause of death assignment by computer-coded verbal autopsy methods and physician coding of verbal autopsy interviews in South Africa

BACKGROUND

The South African national cause of death validation (NCODV 2017/18) project collected a national sample of verbal autopsies (VA) with cause of death (COD) assignment by physician-coded VA (PCVA) and computer-coded VA (CCVA).

OBJECTIVE

The performance of three CCVA algorithms (InterVA-5, InSilicoVA and Tariff 2.0) in assigning a COD was compared with PCVA (reference standard).

METHODS

Seven performance metrics assessed individual and population level agreement of COD assignment by age, sex and place of death subgroups. Positive predictive value (PPV), sensitivity, overall agreement, kappa, and chance corrected concordance (CCC) assessed individual level agreement. Cause-specific mortality fraction (CSMF) accuracy and Spearman's rank correlation assessed population level agreement.

RESULTS

A total of 5386 VA records were analysed. PCVA and CCVAs all identified HIV/AIDS as the leading COD. CCVA PPV and sensitivity, based on confidence intervals, were comparable except for HIV/AIDS, TB, maternal, diabetes mellitus, other cancers, and some injuries. CCVAs performed well for identifying perinatal deaths, road traffic accidents, suicide and homicide but poorly for pneumonia, other infectious diseases and renal failure. Overall agreement between CCVAs and PCVA for the top single cause (48.2-51.6) indicated comparable weak agreement between methods. Overall agreement, for the top three causes showed moderate agreement for InterVA (70.9) and InSilicoVA (73.8). Agreement based on kappa (-0.05-0.49)and CCC (0.06-0.43) was weak to none for all algorithms and groups. CCVAs had moderate to strong agreement for CSMF accuracy, with InterVA-5 highest for neonates (0.90), Tariff 2.0 highest for adults (0.89) and males (0.84), and InSilicoVA highest for females (0.88), elders (0.83) and out-of-facility deaths (0.85). Rank correlation indicated moderate agreement for adults (0.75-0.79).

CONCLUSIONS

Whilst CCVAs identified HIV/AIDS as the leading COD, consistent with PCVA, there is scope for improving the algorithms for use in South Africa.

Evaluation of methods for assigning causes of death from verbal autopsies in India

Background

Physician-coded verbal autopsy (PCVA) is the most widely used method to determine causes of death (COD) in countries where medical certification of death is low. Computer-coded verbal autopsy (CCVA), an alternative method to PCVA for assigning the COD is considered to be efficient and cost-effective. However, the performance of CCVA as compared to PCVA is yet to be established in the Indian context.

Methods

We evaluated the performance of PCVA and three CCVA methods i.e., InterVA 5, InSilico, and Tariff 2.0 on verbal autopsies done using the WHO 2016 VA tool on 2,120 reference standard cases developed from five tertiary care hospitals of Delhi. PCVA methodology involved dual independent review with adjudication, where required. Metrics to assess performance were Cause Specific Mortality Fraction (CSMF), sensitivity, positive predictive value (PPV), CSMF Accuracy, and Kappa statistic.

Results

In terms of the measures of the overall performance of COD assignment methods, for CSMF Accuracy, the PCVA method achieved the highest score of 0.79, followed by 0.67 for Tariff_2.0, 0.66 for Inter-VA and 0.62 for InSilicoVA. The PCVA method also achieved the highest agreement (57%) and Kappa scores (0.54). The PCVA method showed the highest sensitivity for 15 out of 20 causes of death.

Conclusion

Our study found that the PCVA method had the best performance out of all the four COD assignment methods that were tested in our study sample. In order to improve the performance of CCVA methods, multicentric studies with larger sample sizes need to be conducted using the WHO VA tool.

Validation studies of verbal autopsy methods: a systematic review

BACKGROUND

Verbal autopsy (VA) has emerged as an increasingly popular technique to assign cause of death in parts of the world where the majority of deaths occur without proper medical certification. The purpose of this study was to examine the key characteristics of studies that have attempted to validate VA cause of death against an established cause of death.

METHODS

A systematic review was conducted by searching the MEDLINE, EMBASE, Cochrane-library, and Scopus electronic databases. Included studies contained 1) a VA component, 2) a validation component, and 3) original analysis or re-analysis. Characteristics of VA studies were extracted. A total of 527 studies were assessed, and 481 studies screened to give 66 studies selected for data extraction.

RESULTS

Sixty-six studies were included from multiple countries. Ten studies used an existing database. Sixteen studies used the World Health Organization VA questionnaire and 5 studies used the Population Health Metrics Research Consortium VA questionnaire. Physician certification was used in 36 studies and computer coded methods were used in 14 studies. Thirty-seven studies used high level comparator data with detailed laboratory investigations.

CONCLUSION

Most studies found VA to be an effective cause of death assignment method and compared VA cause of death to a high-quality established cause of death. Nonetheless, there were inconsistencies in the methodologies of the validation studies, and many used poor quality comparison cause of death data. Future VA validation studies should adhere to consistent methodological criteria so that policymakers can easily interpret the findings to select the most appropriate VA method.

PROSPERO REGISTRATION

CRD42020186886.

Performance evaluation of machine learning and Computer Coded Verbal Autopsy (CCVA) algorithms for cause of death determination: A comparative analysis of data from rural South Africa

Computer Coded Verbal Autopsy (CCVA) algorithms are commonly used to determine the cause of death (CoD) from questionnaire responses extracted from verbal autopsies (VAs). However, they can only operate on structured data and cannot effectively harness information from unstructured VA narratives. Machine Learning (ML) algorithms have also been applied successfully in determining the CoD from VA narratives, allowing the use of auxiliary information that CCVA algorithms cannot directly utilize. However, most ML-based studies only use responses from the structured questionnaire, and the results lack generalisability and comparability across studies. We present a comparative performance evaluation of ML methods and CCVA algorithms on South African VA narratives data, using data from Agincourt Health and Demographic Surveillance Site (HDSS) with physicians' classifications as the gold standard. The data were collected from 1993 to 2015 and have 16,338 cases. The random forest and extreme gradient boosting classifiers outperformed the other classifiers on the combined dataset, attaining accuracy of 96% respectively, with significant statistical differences in algorithmic performance (p < 0.0001). All our models attained Area Under Receiver Operating Characteristics (AUROC) of greater than 0.884. The InterVA CCVA attained 83% Cause Specific Mortality Fraction accuracy and an Overall Chance-Corrected Concordance of 0.36. We demonstrate that ML models could accurately determine the cause of death from VA narratives. Additionally, through mortality trends and pattern analysis, we discovered that in the first decade of the civil registration system in South Africa, the average life expectancy was approximately 50 years. However, in the second decade, life expectancy significantly dropped, and the population was dying at a much younger average age of 40 years, mostly from the leading HIV related causes. Interestingly, in the third decade, we see a gradual improvement in life expectancy, possibly attributed to effective health intervention programmes. Through a structure and semantic analysis of narratives where experts disagree, we also demonstrate the most frequent terms of traditional healer consultations and visits. The comparative approach also makes this study a baseline that can be used for future research enforcing generalization and comparability. Future study will entail exploring deep learning models for CoD classification.

Validation of physician certified verbal autopsy using conventional autopsy: a large study of adult non-external causes of death in a metropolitan area in Brazil

Background

Reliable mortality data are essential for the development of public health policies. In Brazil, although there is a well-consolidated universal system for mortality data, the quality of information on causes of death (CoD) is not even among Brazilian regions, with a high proportion of ill-defined CoD. Verbal autopsy (VA) is an alternative to improve mortality data. This study aimed to evaluate the performance of an adapted and reduced version of VA in identifying the underlying causes of non-forensic deaths, in São Paulo, Brazil. This is the first time that a version of the questionnaire has been validated considering the autopsy as the gold standard.

Methods

The performance of a physician-certified verbal autopsy (PCVA) was evaluated considering conventional autopsy (macroscopy plus microscopy) as gold standard, based on a sample of 2060 decedents that were sent to the Post-Mortem Verification Service (SVOC-USP). All CoD, from the underlying to the immediate, were listed by both parties, and ICD-10 attributed by a senior coder. For each cause, sensitivity and chance corrected concordance (CCC) were computed considering first the underlying causes attributed by the pathologist and PCVA, and then any CoD listed in the death certificate given by PCVA. Cause specific mortality fraction accuracy (CSMF-accuracy) and chance corrected CSMF-accuracy were computed to evaluate the PCVA performance at the populational level.

Results

There was substantial variability of the sensitivities and CCC across the causes. Well-known chronic diseases with accurate diagnoses that had been informed by physicians to family members, such as various cancers, had sensitivities above 40% or 50%. However, PCVA was not effective in attributing Pneumonia, Cardiomyopathy and Leukemia/Lymphoma as underlying CoD. At populational level, the PCVA estimated cause specific mortality fractions (CSMF) may be considered close to the fractions pointed by the gold standard. The CSMF-accuracy was 0.81 and the chance corrected CSMF-accuracy was 0.49.

Conclusions

The PCVA was efficient in attributing some causes individually and proved effective in estimating the CSMF, which indicates that the method is useful to establish public health priorities.

Assessment of the quality of the vital registration system for under-5 mortality in Yucatan, Mexico

Introduction

Vital registration is an important element in health information systems which can inform policy and strengthen health systems. Mexico has a well-functioning vital registration system; however, there is still room for improvement, especially for deaths of children under 5. This study assesses the quality of the vital registration system in capturing deaths and evaluates the quality of cause of death certification in under-5 deaths in Yucatan, Mexico.

Methods

We collected information on under-5 deaths that occurred in 2015 and 2016 in Yucatan, Mexico. We calculated the Vital Statistics Performance Index (VSPI) to have a general assessment of the vital registration performance. We examined the agreement between vital registration records and medical records at the individual and population levels using the chance-corrected concordance (CCC) and cause-specific mortality fraction (CSMF) accuracy as quality metrics.

Results

We identified 966 records from the vital registry for all under-5 deaths, and 390 were linked to medical records of deaths occurring at public hospitals. The Yucatan vital registration system captured 94.8% of the expected under-5 deaths, with an overall VSPI score of 87.2%. Concordance between underlying cause of death listed in the vital registry and the cause determined by the medical record review varied substantially across causes, with a mean overall chance-corrected concordance across causes of 6.9% for neonates and 46.9% for children. Children had the highest concordance for digestive diseases, and neonates had the highest concordance for meningitis/sepsis. At the population level, the CSMF accuracy for identifying the underlying cause listed was 35.3% for neonates and 67.7% for children.

Conclusions

Although the vital registration system has overall good performance, there are still problems in information about causes of death for children under 5 that are related mostly to certification of the causes of death. The accuracy of information can vary substantially across age groups and causes, with causes reported for neonates being generally less reliable than those for older children. Results highlight the need to implement strategies to improve the certification of causes of death in this population.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12963-022-00284-5.

Validation of SmartVA using conventional autopsy: A study of adult deaths in Brazil

BACKGROUND

Accurate cause of death data are essential to guide health policy. However, mortality surveillance is limited in many low-income countries. In such settings, verbal autopsy (VA) is increasingly used to provide population-level cause of death data. VAs are now widely interpreted using the automated algorithms SmartVA and InterVA. Here we use conventional autopsy as the gold standard to validate SmartVA methodology.

METHODS

This study included adult deaths from natural causes in São Paulo and Recife for which conventional autopsy was indicated. VA was conducted with a relative of the deceased using an amended version of the SmartVA instrument to suit the local context. Causes of death from VA were produced using the SmartVA-Analyze program. Physician coded verbal autopsy (PCVA), conducted on the same questionnaires, and Global Burden of Disease Study data were used as additional comparators. Cause of death data were grouped into 10 broad causes for the validation due to the real-world utility of VA lying in identifying broad population cause of death patterns.

FINDINGS

The study included 2,060 deaths in São Paulo and 1,079 in Recife. The cause specific mortality fractions (CSMFs) estimated using SmartVA were broadly similar to conventional autopsy for: cardiovascular diseases (46.8% vs 54.0%, respectively), cancers (10.6% vs 11.4%), infections (7.0% vs 10.4%) and chronic respiratory disease (4.1% vs 3.7%), causes accounting for 76.1% of the autopsy dataset. The SmartVA CSMF estimates were lower than autopsy for "Other NCDs" (7.8% vs 14.6%) and higher for diabetes (13.0% vs 6.6%). CSMF accuracy of SmartVA compared to autopsy was 84.5%. CSMF accuracy for PCVA was 93.0%.

INTERPRETATION

The results suggest that SmartVA can, with reasonable accuracy, predict the broad cause of death groups important to assess a population's epidemiological transition. VA remains a useful tool for understanding causes of death where medical certification is not possible.

The impact of errors in medical certification on the accuracy of the underlying cause of death

BACKGROUND

Correct certification of cause of death by physicians (i.e. completing the medical certificate of cause of death or MCCOD) and correct coding according to International Classification of Diseases (ICD) rules are essential to produce quality mortality statistics to inform health policy. Despite clear guidelines, errors in medical certification are common. This study objectively measures the impact of different medical certification errors upon the selection of the underlying cause of death.

METHODS

A sample of 1592 error-free MCCODs were selected from the 2017 United States multiple cause of death data. The ten most common types of errors in completing the MCCOD (according to published studies) were individually simulated on the error-free MCCODs. After each simulation, the MCCODs were coded using Iris automated mortality coding software. Chance-corrected concordance (CCC) was used to measure the impact of certification errors on the underlying cause of death. Weights for each error type and Socio-demographic Index (SDI) group (representing different mortality conditions) were calculated from the CCC and categorised (very high, high, medium and low) to describe their effect on cause of death accuracy.

FINDINGS

The only very high impact error type was reporting an ill-defined condition as the underlying cause of death. High impact errors were found to be reporting competing causes in Part 1 [of the death certificate] and illegibility, with medium impact errors being reporting underlying cause in Part 2 [of the death certificate], incorrect or absent time intervals and reporting contributory causes in Part 1, and low impact errors comprising multiple causes per line and incorrect sequence. There was only small difference in error importance between SDI groups.

CONCLUSIONS

Reporting an ill-defined condition as the underlying cause of death can seriously affect the coding outcome, while other certification errors were mitigated through the correct application of mortality coding rules. Training of physicians in not reporting ill-defined conditions on the MCCOD and mortality coders in correct coding practices and using Iris should be important components of national strategies to improve cause of death data quality.

Bayesian hierarchical factor regression models to infer cause of death from verbal autopsy data

In low-resource settings where vital registration of death is not routine it is often of critical interest to determine and study the cause of death (COD) for individuals and the cause-specific mortality fraction (CSMF) for populations. Post-mortem autopsies, considered the gold standard for COD assignment, are often difficult or impossible to implement due to deaths occurring outside the hospital, expense, and/or cultural norms. For this reason, Verbal Autopsies (VAs) are commonly conducted, consisting of a questionnaire administered to next of kin recording demographic information, known medical conditions, symptoms, and other factors for the decedent. This article proposes a novel class of hierarchical factor regression models that avoid restrictive assumptions of standard methods, allow both the mean and covariance to vary with COD category, and can include covariate information on the decedent, region, or events surrounding death. Taking a Bayesian approach to inference, this work develops an MCMC algorithm and validates the FActor Regression for Verbal Autopsy (FARVA) model in simulation experiments. An application of FARVA to real VA data shows improved goodness-of-fit and better predictive performance in inferring COD and CSMF over competing methods. Code and a user manual are made available at https://github.com/kelrenmor/farva.

Limitations to current methods to estimate cause of death: a validation study of a verbal autopsy model

Background: Accurate information on causes of death (CoD) is essential to estimate burden of disease, track global progress, prioritize cost-effective interventions, and inform policies to reduce mortality. In low-income settings, where a significant proportion of deaths take place at home or in poorly-resourced peripheral health facilities, data on CoD often relies on verbal autopsies (VAs). Validations of VAs have been performed against clinical diagnosis, but never before against an acceptable gold standard: the complete diagnostic autopsy (CDA). Methods: We have validated a computer-coded verbal autopsy method -the InterVA- using individual and population metrics to determine CoD against the CDA, in 316 deceased patients of different age groups who died in a tertiary-level hospital in Maputo, Mozambique between 2013 and 2015.   Results: We found a low agreement of the model across all age groups at the individual (kappa statistic ranging from -0.030 to 0.232, lowest in stillbirths and highest in adults) and population levels (chance-corrected cause-specific mortality fraction accuracy ranging from -1.00 to 0.62, lowest in stillbirths, highest in children). The sensitivity in identifying infectious diseases was low (0% for tuberculosis, diarrhea, and disseminated infections, 32% for HIV-related infections, 33% for malaria and 36% for pneumonia). Of maternal deaths, 26 were assigned to eclampsia but only four patients actually died of eclampsia. Conclusions: These findings do not lead to building confidence in current estimates of CoD. They also call to the need to implement autopsy methods where they may be feasible, and to improve the quality and performance of current VA techniques.

Limitations to current methods to estimate cause of death: a validation study of a verbal autopsy model

Background: Accurate information on causes of death (CoD) is essential to estimate burden of disease, track global progress, prioritize cost-effective interventions, and inform policies to reduce mortality. In low-income settings, where a significant proportion of deaths take place at home or in poorly-resourced peripheral health facilities, data on CoD often relies on verbal autopsies (VAs). Validations of VAs have been performed against clinical diagnosis, but never before against an acceptable gold standard: the complete diagnostic autopsy (CDA). Methods: We have validated a computer-coded verbal autopsy method -the InterVA- using individual and population metrics to determine CoD against the CDA, in 316 deceased patients of different age groups who died in a tertiary-level hospital in Maputo, Mozambique between 2013 and 2015.   Results: We found a low agreement of the model across all age groups at the individual (kappa statistic ranging from -0.030 to 0.232, lowest in stillbirths and highest in adults) and population levels (chance-corrected cause-specific mortality fraction accuracy ranging from -1.00 to 0.62, lowest in stillbirths, highest in children). The sensitivity in identifying infectious diseases was low (0% for tuberculosis, diarrhea, and disseminated infections, 32% for HIV-related infections, 33% for malaria and 36% for pneumonia). Of maternal deaths, 26 were assigned to eclampsia but only four patients actually died of eclampsia. Conclusions: These findings do not lead to building confidence in current estimates of CoD. They also call to the need to implement autopsy methods where they may be feasible, and to improve the quality and performance of current VA techniques.

Limitations to current methods to estimate cause of death: a validation study of a verbal autopsy model

Background: Accurate information on causes of death (CoD) is essential to estimate burden of disease, track global progress, prioritize cost-effective interventions, and inform policies to reduce mortality. In low-income settings, where a significant proportion of deaths take place at home or in poorly-resourced peripheral health facilities, data on CoD often relies on verbal autopsies (VAs). Validations of VAs have been performed against clinical diagnosis, but never before against an acceptable gold standard: the complete diagnostic autopsy (CDA).

Methods: We have validated a computer-coded verbal autopsy method –the InterVA- using individual and population metrics to determine CoD against the CDA, in 316 deceased patients of different age groups who died in a tertiary-level hospital in Maputo, Mozambique between 2013 and 2015.  

Results: We found a low agreement of the model across all age groups at the individual (kappa statistic ranging from -0.030 to 0.232, lowest in stillbirths and highest in adults) and population levels (chance-corrected cause-specific mortality fraction accuracy ranging from -1.00 to 0.62, lowest in stillbirths, highest in children). The sensitivity in identifying infectious diseases was low (0% for tuberculosis, diarrhea, and disseminated infections, 32% for HIV-related infections, 33% for malaria and 36% for pneumonia). Of maternal deaths, 26 were assigned to eclampsia but only four patients actually died of eclampsia.

Conclusions: These findings do not lead to building confidence in current estimates of CoD. They also call to the need to implement autopsy methods where they may be feasible, and to improve the quality and performance of current VA techniques.

Using Bayesian Latent Gaussian Graphical Models to Infer Symptom Associations in Verbal Autopsies

Learning dependence relationships among variables of mixed types provides insights in a variety of scientific settings and is a well-studied problem in statistics. Existing methods, however, typically rely on copious, high quality data to accurately learn associations. In this paper, we develop a method for scientific settings where learning dependence structure is essential, but data are sparse and have a high fraction of missing values. Specifically, our work is motivated by survey-based cause of death assessments known as verbal autopsies (VAs). We propose a Bayesian approach to characterize dependence relationships using a latent Gaussian graphical model that incorporates informative priors on the marginal distributions of the variables. We demonstrate such information can improve estimation of the dependence structure, especially in settings with little training data. We show that our method can be integrated into existing probabilistic cause-of-death assignment algorithms and improves model performance while recovering dependence patterns between symptoms that can inform efficient questionnaire design in future data collection.

Using health facility deaths to estimate population causes of neonatal and child mortality in four African countries

BACKGROUND

Verbal autopsy is the main method used in countries with weak civil registration systems for estimating community causes of neonatal and 1-59-month-old deaths. However, validation studies of verbal autopsy methods are limited and assessment has been dependent on hospital-based studies, with uncertain implications for its validity in community settings. If the distribution of community deaths by cause was similar to that of facility deaths, or could be adjusted according to related demographic factors, then the causes of facility deaths could be used to estimate population causes.

METHODS

Causes of neonatal and 1-59-month-old deaths from verbal/social autopsy (VASA) surveys in four African countries were estimated using expert algorithms (EAVA) and physician coding (PCVA). Differences between facility and community deaths in individual causes and cause distributions were examined using chi-square and cause-specific mortality fractions (CSMF) accuracy, respectively. Multinomial logistic regression and random forest models including factors from the VASA studies that are commonly available in Demographic and Health Surveys were built to predict population causes from facility deaths.

RESULTS

Levels of facility and community deaths in the four countries differed for one to four of 10 EAVA or PCVA neonatal causes and zero to three of 12 child causes. CSMF accuracy for facility compared to community deaths in the four countries ranged from 0.74 to 0.87 for neonates and 0.85 to 0.95 for 1-59-month-olds. Crude CSMF accuracy in the prediction models averaged 0.86 to 0.88 for neonates and 0.93 for 1-59-month-olds. Adjusted random forest prediction models increased average CSMF accuracy for neonates to, at most, 0.90, based on small increases in all countries.

CONCLUSIONS

There were few differences in facility and community causes of neonatal and 1-59-month-old deaths in the four countries, and it was possible to project the population CSMF from facility deaths with accuracy greater than the validity of verbal autopsy diagnoses. Confirmation of these findings in additional settings would warrant research into how medical causes of deaths in a representative sample of health facilities can be utilized to estimate the population causes of child death.

Born to fail: flaws in replication design produce intended results

We recently published in BMC Medicine an evaluation of the comparative diagnostic performance of InSilicoVA, a software to map the underlying causes of death from verbal autopsy interviews. The developers of this software claim to have failed to replicate our results and appear to have also failed to locate our replication archive for this work. In this Correspondence, we provide feedback on how this might have been done more usefully and offer some suggestions to improve future attempts at reproducible research. We also offer an alternative interpretation of the results presented by Li et al., namely that, out of 100 verbal autopsy interviews, InSilicoVA will, at best, correctly identify the underlying cause of death in 40 cases and incorrectly in 60 - a markedly inferior performance to alternative existing approaches.

Regularized Bayesian transfer learning for population-level etiological distributions

Computer-coded verbal autopsy (CCVA) algorithms predict cause of death from high-dimensional family questionnaire data (verbal autopsy) of a deceased individual, which are then aggregated to generate national and regional estimates of cause-specific mortality fractions. These estimates may be inaccurate if CCVA is trained on non-local training data different from the local population of interest. This problem is a special case of transfer learning, i.e., improving classification within a target domain (e.g., a particular population) with the classifier trained in a source-domain. Most transfer learning approaches concern individual-level (e.g., a person’s) classification. Social and health scientists such as epidemiologists are often more interested with understanding etiological distributions at the population-level. The sample sizes of their data sets are typically orders of magnitude smaller than those used for common transfer learning applications like image classification, document identification, etc. We present a parsimonious hierarchical Bayesian transfer learning framework to directly estimate population-level class probabilities in a target domain, using any baseline classifier trained on source-domain, and a small labeled target-domain dataset. To address small sample sizes, we introduce a novel shrinkage prior for the transfer error rates guaranteeing that, in absence of any labeled target-domain data or when the baseline classifier is perfectly accurate, our transfer learning agrees with direct aggregation of predictions from the baseline classifier, thereby subsuming the default practice as a special case. We then extend our approach to use an ensemble of baseline classifiers producing an unified estimate. Theoretical and empirical results demonstrate how the ensemble model favors the most accurate baseline classifier. We present data analyses demonstrating the utility of our approach.

Robustness of the Tariff method for diagnosing verbal autopsies: impact of additional site data on the relationship between symptom and cause

Background

Verbal autopsy (VA) is increasingly being considered as a cost-effective method to improve cause of death information in countries with low quality vital registration. VA algorithms that use empirical data have an advantage over expert derived algorithms in that they use responses to the VA instrument as a reference instead of physician opinion. It is unclear how stable these data driven algorithms, such as the Tariff 2.0 method, are to cultural and epidemiological variations in populations where they might be employed.

Methods

VAs were conducted in three sites as part of the Improving Methods to Measure Comparable Mortality by Cause (IMMCMC) study: Bohol, Philippines; Chandpur and Comila Districts, Bangladesh; and Central and Eastern Highlands Provinces, Papua New Guinea. Similar diagnostic criteria and cause lists as the Population Health Metrics Research Consortium (PHMRC) study were used to identify gold standard (GS) deaths. We assessed changes in Tariffs by examining the proportion of Tariffs that changed significantly after the addition of the IMMCMC dataset to the PHMRC dataset.

Results

The IMMCMC study added 3512 deaths to the GS VA database (2491 adults, 320 children, and 701 neonates). Chance-corrected cause specific mortality fractions for Tariff improved with the addition of the IMMCMC dataset for adults (+ 5.0%), children (+ 5.8%), and neonates (+ 1.5%). 97.2% of Tariffs did not change significantly after the addition of the IMMCMC dataset.

Conclusions

Tariffs generally remained consistent after adding the IMMCMC dataset. Population level performance of the Tariff method for diagnosing VAs improved marginally for all age groups in the combined dataset. These findings suggest that cause-symptom relationships of Tariff 2.0 might well be robust across different population settings in developing countries. Increasing the total number of GS deaths improves the validity of Tariff and provides a foundation for the validation of other empirical algorithms.

Validation of verbal autopsy methods for assessment of child mortality in sub-Saharan Africa and the policy implication: a rapid review

Reliable data on the cause of child death is the cornerstone for evidence-informed health policy making towards improving child health outcomes. Unfortunately, accurate data on cause of death is essentially lacking in most countries of sub-Saharan Africa due to the widespread absence of functional Civil Registration and Vital Statistics (CRVS) systems. To address this problem, verbal autopsy (VA) has gained prominence as a strategy for obtaining Cause of Death (COD) information in populations where CRVS are absent. This study reviewed publications that investigated the validation of VA methods for assessment of COD. A MEDLINE PubMed search was undertaken in June 2018 for studies published in English that investigated the validation of VA methods in sub-Saharan Africa from 1990-2018. Of the 17 studies identified, 9 fulfilled the study inclusion criteria from which additional five relevant studies were found by reviewing their references. The result showed that Physician-Certified Verbal Autopsy (PCVA) was the most widely used VA method. Validation studies comparing PCVA to hospital records, expert algorithm and InterVA demonstrated mixed and highly varied outcomes. The accuracy and reliability of the VA methods depended on level of healthcare the respondents have access to and the knowledge of the physicians on the local disease aetiology and epidemiology. As the countries in sub-Saharan Africa continue to battle with dysfunctional CRVS system, VA will remain the only viable option for the supply of child mortality data necessary for policy making.

Automatically determining cause of death from verbal autopsy narratives

BACKGROUND

A verbal autopsy (VA) is a post-hoc written interview report of the symptoms preceding a person's death in cases where no official cause of death (CoD) was determined by a physician. Current leading automated VA coding methods primarily use structured data from VAs to assign a CoD category. We present a method to automatically determine CoD categories from VA free-text narratives alone.

METHODS

After preprocessing and spelling correction, our method extracts word frequency counts from the narratives and uses them as input to four different machine learning classifiers: naïve Bayes, random forest, support vector machines, and a neural network.

RESULTS

For individual CoD classification, our best classifier achieves a sensitivity of.770 for adult deaths for 15 CoD categories (as compared to the current best reported sensitivity of.57), and.662 with 48 WHO categories. When predicting the CoD distribution at the population level, our best classifier achieves.962 cause-specific mortality fraction accuracy for 15 categories and.908 for 48 categories, which is on par with leading CoD distribution estimation methods.

CONCLUSIONS

Our narrative-based machine learning classifier performs as well as classifiers based on structured data at the individual level. Moreover, our method demonstrates that VA narratives provide important information that can be used by a machine learning system for automated CoD classification. Unlike the structured questionnaire-based methods, this method can be applied to any verbal autopsy dataset, regardless of the collection process or country of origin.

External injuries, trauma and avoidable deaths in Agincourt, South Africa: a retrospective observational and qualitative study

OBJECTIVE

Injury burden is highest in low-income and middle-income countries. To reduce avoidable deaths, it is necessary to identify health system deficiencies preventing timely, quality care. We developed criteria to use verbal autopsy (VA) data to identify avoidable deaths and associated health system deficiencies.

SETTING

Agincourt, a rural Bushbuckridge municipality, Mpumalanga Province, South Africa.

PARTICIPANTS

Agincourt Health and Socio-Demographic Surveillance System and healthcare providers (HCPs) from local hospitals.

METHODS

A literature review to explore definitions of avoidable deaths after trauma and barriers to access to care using the 'three delays framework' (seeking, reaching and receiving care) was performed. Based on these definitions, this study developed criteria, applicable for use with VA data, for identifying avoidable death and which of the three delays contributed to avoidable deaths. These criteria were then applied retrospectively to the VA-defined category external injury deaths (EIDs-a subset of which are trauma deaths) from 2012 to 2015. The findings were validated by external expert review. Key informant interviews (KIIs) with HCPs were performed to further explore delays to care.

RESULTS

Using VA data, avoidable death was defined with a focus on survivability, using level of consciousness at the scene and ability to seek care as indicators. Of 260 EIDs (189 trauma deaths), there were 104 (40%) avoidable EIDs and 78 (30%) avoidable trauma deaths (41% of trauma deaths). Delay in receiving care was the largest contributor to avoidable EIDs (61%) and trauma deaths (59%), followed by delay in seeking care (24% and 23%) and in reaching care (15% and 18%). KIIs revealed context-specific factors contributing to the third delay, including difficult referral systems.

CONCLUSIONS

A substantial proportion of EIDs and trauma deaths were avoidable, mainly occurring due to facility-based delays in care. Interventions, including strengthening referral networks, may substantially reduce trauma deaths.

Monitoring progress in reducing maternal mortality using verbal autopsy methods in vital registration systems: what can we conclude about specific causes of maternal death?

Reducing maternal mortality is a key focus of development strategies and one of the indicators used to measure progress towards achieving the Sustainable Development Goals. In the absence of medical certification of the cause of deaths that occur in the community, verbal autopsy (VA) methods are the only available means to assess levels and trends of maternal deaths that occur outside health facilities. The 2016 World Health Organization VA Instrument facilitates the identification of eight specific causes of maternal death, yet maternal deaths are often unsupervised, leading to sparse and generally poor symptom reporting to inform a reliable diagnosis using VAs. There is little research evidence to support the reliable identification of specific causes of maternal death in the context of routine VAs. We recommend that routine VAs are only used to capture the event of a maternal death and that more detailed follow-up interviews are used to identify the specific causes.

Automated verbal autopsy classification: using one-against-all ensemble method and Naïve Bayes classifier

Verbal autopsy (VA) deals with post-mortem surveys about deaths, mostly in low and middle income countries, where the majority of deaths occur at home rather than a hospital, for retrospective assignment of causes of death (COD) and subsequently evidence-based health system strengthening. Automated algorithms for VA COD assignment have been developed and their performance has been assessed against physician and clinical diagnoses. Since the performance of automated classification methods remains low, we aimed to enhance the Naïve Bayes Classifier (NBC) algorithm to produce better ranked COD classifications on 26,766 deaths from four globally diverse VA datasets compared to some of the leading VA classification methods, namely Tariff, InterVA-4, InSilicoVA and NBC. We used a different strategy, by training multiple NBC algorithms using the one-against-all approach (OAA-NBC). To compare performance, we computed the cumulative cause-specific mortality fraction (CSMF) accuracies for population-level agreement from rank one to five COD classifications. To assess individual-level COD assignments, cumulative partially-chance corrected concordance (PCCC) and sensitivity was measured for up to five ranked classifications. Overall results show that OAA-NBC consistently assigns CODs that are the most alike physician and clinical COD assignments compared to some of the leading algorithms based on the cumulative CSMF accuracy, PCCC and sensitivity scores. The results demonstrate that our approach improves the performance of classification (sensitivity) by between 6% and 8% compared with other VA algorithms. Population-level agreements for OAA-NBC and NBC were found to be similar or higher than the other algorithms used in the experiments. Although OAA-NBC still requires improvement for individual-level COD assignment, the one-against-all approach improved its ability to assign CODs that more closely resemble physician or clinical COD classifications compared to some of the other leading VA classifiers.

Automated verbal autopsy classification: using one-against-all ensemble method and Naïve Bayes classifier

Verbal autopsy (VA) deals with post-mortem surveys about deaths, mostly in low and middle income countries, where the majority of deaths occur at home rather than a hospital, for retrospective assignment of causes of death (COD) and subsequently evidence-based health system strengthening. Automated algorithms for VA COD assignment have been developed and their performance has been assessed against physician and clinical diagnoses. Since the performance of automated classification methods remains low, we aimed to enhance the Naïve Bayes Classifier (NBC) algorithm to produce better ranked COD classifications on 26,766 deaths from four globally diverse VA datasets compared to some of the leading VA classification methods, namely Tariff, InterVA-4, InSilicoVA and NBC. We used a different strategy, by training multiple NBC algorithms using the one-against-all approach (OAA-NBC). To compare performance, we computed the cumulative cause-specific mortality fraction (CSMF) accuracies for population-level agreement from rank one to five COD classifications. To assess individual-level COD assignments, cumulative partially-chance corrected concordance (PCCC) and sensitivity was measured for up to five ranked classifications. Overall results show that OAA-NBC consistently assigns CODs that are the most alike physician and clinical COD assignments compared to some of the leading algorithms based on the cumulative CSMF accuracy, PCCC and sensitivity scores. The results demonstrate that our approach improves the performance of classification (sensitivity) by between 6% and 8% compared with other VA algorithms. Population-level agreements for OAA-NBC and NBC were found to be similar or higher than the other algorithms used in the experiments. Although OAA-NBC still requires improvement for individual-level COD assignment, the one-against-all approach improved its ability to assign CODs that more closely resemble physician or clinical COD classifications compared to some of the other leading VA classifiers.

Assessing the quality of medical death certification: a case study of concordance between national statistics and results from a medical record review in a regional hospital in the Philippines

Background

Medical certificates of cause of death (MCCOD) issued by hospital physicians are a key input to vital registration systems. Deaths certified by hospital physicians have been implicitly considered to be of high quality, but recent evidence suggests otherwise. We conducted a medical record review (MRR) of hospital MCCOD in the Philippines and compared the cause of death concordance with certificates coded by the Philippines Statistics Authority (PSA).

Methods

MCCOD for adult deaths in Bohol Regional Hospital (BRH) in 2007–2008 and 2011 were collected and reviewed by a team of study physicians. Corresponding MCCOD coded by the PSA were linked by a hospital identifier. The study physicians wrote a new MCCOD using the patient medical record, noted the quality of the medical record to produce a cause of death, and indicated whether it was necessary to change the underlying cause of death (UCOD). Chance-corrected concordance, cause-specific mortality fraction (CSMF) accuracy, and chance-corrected CSMF were used to examine the concordance between the MRR and PSA.

Results

A total of 1052 adult deaths were linked between the MRR and PSA. Median chance-corrected concordance was 0.73, CSMF accuracy was 0.85, and chance-corrected CSMF accuracy was 0.58. 74.8% of medical records were deemed to be of high enough quality to assign a cause of death, yet study physicians indicated that it was necessary to change the UCOD in 41% of deaths, 82% of which required addition of a new UCOD.

Conclusions

Medical records were generally of sufficient quality to assign a cause of death and concordance between the PSA and MRR was reasonably high, suggesting that routine mortality statistics data are reasonably accurate for describing population level causes of death in Bohol. While overall agreement between the PSA and MRR in major cause groups was sufficient for public health purposes, improvements in death certification practices are recommended to help physicians differentiate between treatable (immediate) COD and COD that are important for public health surveillance.

Electronic supplementary material

The online version of this article (10.1186/s12963-018-0178-0) contains supplementary material, which is available to authorized users.

How useful are registered birth statistics for health and social policy? A global systematic assessment of the availability and quality of birth registration data

Background

The registration and certification of births has a wide array of individual and societal benefits. While near-universal in some parts of the world, birth registration is less common in many low- and middle-income countries, and the quality of vital statistics vary. We assembled publicly available birth registration records for as many countries as possible into a novel global birth registration database, and we present a systematic assessment of available data.

Methods

We obtained 4918 country-years of data from 145 countries covering the period 1948–2015. We compared these to existing estimates of total births to assess completeness of public data and adapted existing methods to evaluate the quality and timeliness of the data.

Results

Since 1980, approximately one billion births were registered and shared in public databases. Compared to estimates of fertility, this represents only 40.0% of total births in the peak year, 2011. Approximately 74 million births (53.1%) per year occur in countries whose systems do not systematically register them and release the aggregate records. Considering data quality, timeliness, and completeness in country-years where data are available, only about 12 million births per year (8.6%) occur in countries with high-performing registration systems.

Conclusions

This analysis highlights the gaps in available data. Our objective and low-cost approach to assessing the performance of birth registration systems can be helpful to monitor country progress, and to help national and international policymakers set targets for strengthening birth registration systems.

Electronic supplementary material

The online version of this article (10.1186/s12963-018-0180-6) contains supplementary material, which is available to authorized users.

Performance of InSilicoVA for assigning causes of death to verbal autopsies: multisite validation study using clinical diagnostic gold standards

BACKGROUND

Recently, a new algorithm for automatic computer certification of verbal autopsy data named InSilicoVA was published. The authors presented their algorithm as a statistical method and assessed its performance using a single set of model predictors and one age group.

METHODS

We perform a standard procedure for analyzing the predictive accuracy of verbal autopsy classification methods using the same data and the publicly available implementation of the algorithm released by the authors. We extend the original analysis to include children and neonates, instead of only adults, and test accuracy using different sets of predictors, including the set used in the original paper and a set that matches the released software.

RESULTS

The population-level performance (i.e., predictive accuracy) of the algorithm varied from 2.1 to 37.6% when trained on data preprocessed similarly as in the original study. When trained on data that matched the software default format, the performance ranged from -11.5 to 17.5%. When using the default training data provided, the performance ranged from -59.4 to -38.5%. Overall, the InSilicoVA predictive accuracy was found to be 11.6-8.2 percentage points lower than that of an alternative algorithm. Additionally, the sensitivity for InSilicoVA was consistently lower than that for an alternative diagnostic algorithm (Tariff 2.0), although the specificity was comparable.

CONCLUSIONS

The default format and training data provided by the software lead to results that are at best suboptimal, with poor cause-of-death predictive performance. This method is likely to generate erroneous cause of death predictions and, even if properly configured, is not as accurate as alternative automated diagnostic methods.

Measuring mortality due to HIV-associated tuberculosis among adults in South Africa: Comparing verbal autopsy, minimally-invasive autopsy, and research data

BACKGROUND

The World Health Organization (WHO) aims to reduce tuberculosis (TB) deaths by 95% by 2035; tracking progress requires accurate measurement of TB mortality. International Classification of Diseases (ICD) codes do not differentiate between HIV-associated TB and HIV more generally. Verbal autopsy (VA) is used to estimate cause of death (CoD) patterns but has mostly been validated against a suboptimal gold standard for HIV and TB. This study, conducted among HIV-positive adults, aimed to estimate the accuracy of VA in ascertaining TB and HIV CoD when compared to a reference standard derived from a variety of clinical sources including, in some, minimally-invasive autopsy (MIA).

METHODS AND FINDINGS

Decedents were enrolled into a trial of empirical TB treatment or a cohort exploring diagnostic algorithms for TB in South Africa. The WHO 2012 instrument was used; VA CoD were assigned using physician-certified VA (PCVA), InterVA-4, and SmartVA-Analyze. Reference CoD were assigned using MIA, research, and health facility data, as available. 259 VAs were completed: 147 (57%) decedents were female; median age was 39 (interquartile range [IQR] 33-47) years and CD4 count 51 (IQR 22-102) cells/μL. Compared to reference CoD that included MIA (n = 34), VA underestimated mortality due to HIV/AIDS (94% reference, 74% PCVA, 47% InterVA-4, and 41% SmartVA-Analyze; chance-corrected concordance [CCC] 0.71, 0.42, and 0.31, respectively) and HIV-associated TB (41% reference, 32% PCVA; CCC 0.23). For individual decedents, all VA methods agreed poorly with reference CoD that did not include MIA (n = 259; overall CCC 0.14, 0.06, and 0.15 for PCVA, InterVA-4, and SmartVA-Analyze); agreement was better at population level (cause-specific mortality fraction accuracy 0.78, 0.61, and 0.57, for the three methods, respectively).

CONCLUSIONS

Current VA methods underestimate mortality due to HIV-associated TB. ICD and VA methods need modifications that allow for more specific evaluation of HIV-related deaths and direct estimation of mortality due to HIV-associated TB.

Prevalence, Incidence, and Mortality of Stroke in China: Results from a Nationwide Population-Based Survey of 480 687 Adults

BACKGROUND

China bears the biggest stroke burden in the world. However, little is known about the current prevalence, incidence, and mortality of stroke at the national level, and the trend in the past 30 years.

METHODS

In 2013, a nationally representative door-to-door survey was conducted in 155 urban and rural centers in 31 provinces in China, totaling 480 687 adults aged ≥20 years. All stroke survivors were considered as prevalent stroke cases at the prevalent time (August 31, 2013). First-ever strokes that occurred during 1 year preceding the survey point-prevalent time were considered as incident cases. According to computed tomography/MRI/autopsy findings, strokes were categorized into ischemic stroke, intracerebral hemorrhage, subarachnoid hemorrhage, and stroke of undetermined type.

RESULTS

Of 480 687 participants, 7672 were diagnosed with a prevalent stroke (1596.0/100 000 people) and 1643 with incident strokes (345.1/100 000 person-years). The age-standardized prevalence, incidence, and mortality rates were 1114.8/100 000 people, 246.8 and 114.8/100 000 person-years, respectively. Pathological type of stroke was documented by computed tomography/MRI brain scanning in 90% of prevalent and 83% of incident stroke cases. Among incident and prevalent strokes, ischemic stroke constituted 69.6% and 77.8%, intracerebral hemorrhage 23.8% and 15.8%, subarachnoid hemorrhage 4.4% and 4.4%, and undetermined type 2.1% and 2.0%, respectively. Age-specific stroke prevalence in men aged ≥40 years was significantly greater than the prevalence in women (P<0.001). The most prevalent risk factors among stroke survivors were hypertension (88%), smoking (48%), and alcohol use (44%). Stroke prevalence estimates in 2013 were statistically greater than those reported in China 3 decades ago, especially among rural residents (P=0.017). The highest annual incidence and mortality of stroke was in Northeast (365 and 159/100 000 person-years), then Central areas (326 and 154/100 000 person-years), and the lowest incidence was in Southwest China (154/100 000 person-years), and the lowest mortality was in South China (65/100 000 person-years) (P<0.002).

CONCLUSIONS

Stroke burden in China has increased over the past 30 years, and remains particularly high in rural areas. There is a north-to-south gradient in stroke in China, with the greatest stroke burden observed in the northern and central regions.

National and subnational all-cause and cause-specific child mortality in China, 1996-2015: a systematic analysis with implications for the Sustainable Development Goals

BACKGROUND

China has achieved Millennium Development Goal 4 to reduce under-5 mortality rate by two-thirds between 1990 and 2015. In this study, we estimated the national and subnational levels and causes of child mortality in China annually from 1996 to 2015 to draw implications for achievement of the SDGs for China and other low-income and middle-income countries.

METHODS

In this systematic analysis, we adjusted empirical data on levels and causes of child mortality collected in the China Maternal and Child Health Surveillance System to generate representative estimates at the national and subnational levels. In adjusting the data, we considered the sampling design and probability, applied smoothing techniques to produce stable trends, fitted livebirth and age-specific death estimates to natvional estimates produced by the UN for international comparison, and partitioned national estimates of infrequent causes produced by independent sources to the subnational level.

FINDINGS

Between 1996 and 2015, the under-5 mortality rate in China declined from 50·8 per 1000 livebirths to 10·7 per 1000 livebirths, at an average annual rate of reduction of 8·2%. However, 181 600 children still died before their fifth birthday, with 93 400 (51·5%) deaths occurring in neonates. Great inequity exists in child mortality across regions and in urban versus rural areas. The leading causes of under-5 mortality in 2015 were congenital abnormalities (35 700 deaths, 95% uncertainty range [UR] 28 400-45 200), preterm birth complications (30 900 deaths, 24 200-40 800), and injuries (26 600 deaths, 21 000-33 400). Pneumonia contributed to a higher proportion of deaths in the western region of China than in the eastern and central regions, and injury was a main cause of death in rural areas. Variations in cause-of-death composition by age were also examined. The contribution of preterm birth complications to mortality decreased after the neonatal period; congenital abnormalities remained an important cause of mortality throughout infancy, whereas the contribution of injuries to mortality increased after the first year of life.

INTERPRETATION

China has achieved a rapid reduction in child mortality in 1996-2015. The decline has been widespread across regions, urban and rural areas, age groups, and cause-of-death categories, but great disparities remain. The western region and rural areas and especially western rural areas should receive most attention in improving child survival through enhanced policy and programmes in the Sustainable Development Goals era. Continued investment is crucial in primary and secondary prevention of deaths due to congenital abnormalities, preterm birth complications, and injuries nationally, and of deaths due to pneumonia in western rural areas. The study also has implications for improving child survival and civil registration and vital statistics in other low-income and middle-income countries.

FUNDING

Bill & Melinda Gates Foundation.

The paradox of verbal autopsy in cause of death assignment: symptom question unreliability but predictive accuracy

Background

We believe that it is important that governments understand the reliability of the mortality data which they have at their disposable to guide policy debates. In many instances, verbal autopsy (VA) will be the only source of mortality data for populations, yet little is known about how the accuracy of VA diagnoses is affected by the reliability of the symptom responses. We previously described the effect of the duration of time between death and VA administration on VA validity. In this paper, using the same dataset, we assess the relationship between the reliability and completeness of symptom responses and the reliability and accuracy of cause of death (COD) prediction.

Methods

The study was based on VAs in the Population Health Metrics Research Consortium (PHMRC) VA Validation Dataset from study sites in Bohol and Manila, Philippines and Andhra Pradesh, India. The initial interview was repeated within 3–52 months of death. Question responses were assessed for reliability and completeness between the two survey rounds. COD was predicted by Tariff Method.

Results

A sample of 4226 VAs was collected for 2113 decedents, including 1394 adults, 349 children, and 370 neonates. Mean question reliability was unexpectedly low (kappa = 0.447): 42.5 % of responses positive at the first interview were negative at the second, and 47.9 % of responses positive at the second had been negative at the first. Question reliability was greater for the short form of the PHMRC instrument (kappa = 0.497) and when analyzed at the level of the individual decedent (kappa = 0.610). Reliability at the level of the individual decedent was associated with COD predictive reliability and predictive accuracy.

Conclusions

Families give coherent accounts of events leading to death but the details vary from interview to interview for the same case. Accounts are accurate but inconsistent; different subsets of symptoms are identified on each occasion. However, there are sufficient accurate and consistent subsets of symptoms to enable the Tariff Method to assign a COD.

Questions which contributed most to COD prediction were also the most reliable and consistent across repeat interviews; these have been included in the short form VA questionnaire. Accuracy and reliability of diagnosis for an individual death depend on the quality of interview. This has considerable implications for the progressive roll out of VAs into civil registration and vital statistics (CRVS) systems.

Electronic supplementary material

The online version of this article (doi:10.1186/s12963-016-0104-2) contains supplementary material, which is available to authorized users.

Validating hierarchical verbal autopsy expert algorithms in a large data set with known causes of death

BACKGROUND

Physician assessment historically has been the most common method of analyzing verbal autopsy (VA) data. Recently, the World Health Organization endorsed two automated methods, Tariff 2.0 and InterVA-4, which promise greater objectivity and lower cost. A disadvantage of the Tariff method is that it requires a training data set from a prior validation study, while InterVA relies on clinically specified conditional probabilities. We undertook to validate the hierarchical expert algorithm analysis of VA data, an automated, intuitive, deterministic method that does not require a training data set.

METHODS

Using Population Health Metrics Research Consortium study hospital source data, we compared the primary causes of 1629 neonatal and 1456 1-59 month-old child deaths from VA expert algorithms arranged in a hierarchy to their reference standard causes. The expert algorithms were held constant, while five prior and one new "compromise" neonatal hierarchy, and three former child hierarchies were tested. For each comparison, the reference standard data were resampled 1000 times within the range of cause-specific mortality fractions (CSMF) for one of three approximated community scenarios in the 2013 WHO global causes of death, plus one random mortality cause proportions scenario. We utilized CSMF accuracy to assess overall population-level validity, and the absolute difference between VA and reference standard CSMFs to examine particular causes. Chance-corrected concordance (CCC) and Cohen's kappa were used to evaluate individual-level cause assignment.

RESULTS

Overall CSMF accuracy for the best-performing expert algorithm hierarchy was 0.80 (range 0.57-0.96) for neonatal deaths and 0.76 (0.50-0.97) for child deaths. Performance for particular causes of death varied, with fairly flat estimated CSMF over a range of reference values for several causes. Performance at the individual diagnosis level was also less favorable than that for overall CSMF (neonatal: best CCC = 0.23, range 0.16-0.33; best kappa = 0.29, 0.23-0.35; child: best CCC = 0.40, 0.19-0.45; best kappa = 0.29, 0.07-0.35).

CONCLUSIONS

Expert algorithms in a hierarchy offer an accessible, automated method for assigning VA causes of death. Overall population-level accuracy is similar to that of more complex machine learning methods, but without need for a training data set from a prior validation study.

Moving from medical to health systems classifications of deaths: extending verbal autopsy to collect information on the circumstances of mortality

BACKGROUND

Verbal autopsy (VA) is a health surveillance technique used in low and middle-income countries to establish medical causes of death (CODs) for people who die outside hospitals and/or without registration. By virtue of the deaths it investigates, VA is also an opportunity to examine social exclusion from access to health systems. The aims were to develop a system to collect and interpret information on social and health systems determinants of deaths investigated in VA.

METHODS

A short set of questions on care pathways, circumstances and events at and around the time of death were developed and integrated into the WHO 2012 short form VA (SF-VA). Data were subsequently analysed from two census rounds in the Agincourt Health and Socio-Demographic Surveillance Site (HDSS), South Africa in 2012 and 2013 where the SF-VA had been applied. InterVA and descriptive analysis were used to calculate cause-specific mortality fractions (CSMFs), and to examine responses to the new indicators and whether and how they varied by medical CODs and age/sex sub-groups.

RESULTS

One thousand two hundred forty-nine deaths were recorded in the Agincourt HDSS censuses in 2012-13 of which 1,196 (96 %) had complete VA data. Infectious and non-communicable conditions accounted for the majority of deaths (47 % and 39 % respectively) with smaller proportions attributed to external, neonatal and maternal causes (5 %, 2 % and 1 % respectively). 5 % of deaths were of indeterminable cause. The new indicators revealed multiple problems with access to care at the time of death: 39 % of deaths did not call for help, 36 % found care unaffordable overall, and 33 % did not go to a facility. These problems were reported consistently across age and sex sub-groups. Acute conditions and younger age groups had fewer problems with overall costs but more with not calling for help or going to a facility. An illustrative health systems interpretation suggests extending and promoting existing provisions for transport and financial access in this setting.

CONCLUSIONS

Supplementing VA with questions on the circumstances of mortality provides complementary information to CSMFs relevant for health planning. Further contextualisation of the method and results are underway with health systems stakeholders to develop the interpretation sequence as part of a health policy and systems research approach.

A shortened verbal autopsy instrument for use in routine mortality surveillance systems

BACKGROUND

Verbal autopsy (VA) is recognized as the only feasible alternative to comprehensive medical certification of deaths in settings with no or unreliable vital registration systems. However, a barrier to its use by national registration systems has been the amount of time and cost needed for data collection. Therefore, a short VA instrument (VAI) is needed. In this paper we describe a shortened version of the VAI developed for the Population Health Metrics Research Consortium (PHMRC) Gold Standard Verbal Autopsy Validation Study using a systematic approach.

METHODS

We used data from the PHMRC validation study. Using the Tariff 2.0 method, we first established a rank order of individual questions in the PHMRC VAI according to their importance in predicting causes of death. Second, we reduced the size of the instrument by dropping questions in reverse order of their importance. We assessed the predictive performance of the instrument as questions were removed at the individual level by calculating chance-corrected concordance and at the population level with cause-specific mortality fraction (CSMF) accuracy. Finally, the optimum size of the shortened instrument was determined using a first derivative analysis of the decline in performance as the size of the VA instrument decreased for adults, children, and neonates.

RESULTS

The full PHMRC VAI had 183, 127, and 149 questions for adult, child, and neonatal deaths, respectively. The shortened instrument developed had 109, 69, and 67 questions, respectively, representing a decrease in the total number of questions of 40-55%. The shortened instrument, with text, showed non-significant declines in CSMF accuracy from the full instrument with text of 0.4%, 0.0%, and 0.6% for the adult, child, and neonatal modules, respectively.

CONCLUSIONS

We developed a shortened VAI using a systematic approach, and assessed its performance when administered using hand-held electronic tablets and analyzed using Tariff 2.0. The length of a VA questionnaire was shortened by almost 50% without a significant drop in performance. The shortened VAI developed reduces the burden of time and resources required for data collection and analysis of cause of death data in civil registration systems.

Improving performance of the Tariff Method for assigning causes of death to verbal autopsies

BACKGROUND

Reliable data on the distribution of causes of death (COD) in a population are fundamental to good public health practice. In the absence of comprehensive medical certification of deaths, the only feasible way to collect essential mortality data is verbal autopsy (VA). The Tariff Method was developed by the Population Health Metrics Research Consortium (PHMRC) to ascertain COD from VA information. Given its potential for improving information about COD, there is interest in refining the method. We describe the further development of the Tariff Method.

METHODS

This study uses data from the PHMRC and the National Health and Medical Research Council (NHMRC) of Australia studies. Gold standard clinical diagnostic criteria for hospital deaths were specified for a target cause list. VAs were collected from families using the PHMRC verbal autopsy instrument including health care experience (HCE). The original Tariff Method (Tariff 1.0) was trained using the validated PHMRC database for which VAs had been collected for deaths with hospital records fulfilling the gold standard criteria (validated VAs). In this study, the performance of Tariff 1.0 was tested using VAs from household surveys (community VAs) collected for the PHMRC and NHMRC studies. We then corrected the model to account for the previous observed biases of the model, and Tariff 2.0 was developed. The performance of Tariff 2.0 was measured at individual and population levels using the validated PHMRC database.

RESULTS

For median chance-corrected concordance (CCC) and mean cause-specific mortality fraction (CSMF) accuracy, and for each of three modules with and without HCE, Tariff 2.0 performs significantly better than the Tariff 1.0, especially in children and neonates. Improvement in CSMF accuracy with HCE was 2.5%, 7.4%, and 14.9% for adults, children, and neonates, respectively, and for median CCC with HCE it was 6.0%, 13.5%, and 21.2%, respectively. Similar levels of improvement are seen in analyses without HCE.

CONCLUSIONS

Tariff 2.0 addresses the main shortcomings of the application of the Tariff Method to analyze data from VAs in community settings. It provides an estimation of COD from VAs with better performance at the individual and population level than the previous version of this method, and it is publicly available for use.

Naive Bayes classifiers for verbal autopsies: comparison to physician-based classification for 21,000 child and adult deaths

BACKGROUND

Verbal autopsies (VA) are increasingly used in low- and middle-income countries where most causes of death (COD) occur at home without medical attention, and home deaths differ substantially from hospital deaths. Hence, there is no plausible "standard" against which VAs for home deaths may be validated. Previous studies have shown contradictory performance of automated methods compared to physician-based classification of CODs. We sought to compare the performance of the classic naive Bayes classifier (NBC) versus existing automated classifiers, using physician-based classification as the reference.

METHODS

We compared the performance of NBC, an open-source Tariff Method (OTM), and InterVA-4 on three datasets covering about 21,000 child and adult deaths: the ongoing Million Death Study in India, and health and demographic surveillance sites in Agincourt, South Africa and Matlab, Bangladesh. We applied several training and testing splits of the data to quantify the sensitivity and specificity compared to physician coding for individual CODs and to test the cause-specific mortality fractions at the population level.

RESULTS

The NBC achieved comparable sensitivity (median 0.51, range 0.48-0.58) to OTM (median 0.50, range 0.41-0.51), with InterVA-4 having lower sensitivity (median 0.43, range 0.36-0.47) in all three datasets, across all CODs. Consistency of CODs was comparable for NBC and InterVA-4 but lower for OTM. NBC and OTM achieved better performance when using a local rather than a non-local training dataset. At the population level, NBC scored the highest cause-specific mortality fraction accuracy across the datasets (median 0.88, range 0.87-0.93), followed by InterVA-4 (median 0.66, range 0.62-0.73) and OTM (median 0.57, range 0.42-0.58).

CONCLUSIONS

NBC outperforms current similar COD classifiers at the population level. Nevertheless, no current automated classifier adequately replicates physician classification for individual CODs. There is a need for further research on automated classifiers using local training and test data in diverse settings prior to recommending any replacement of physician-based classification of verbal autopsies.

Distribution of chronic disease mortality and deterioration in household socioeconomic status in rural Bangladesh: an analysis over a 24-year period

BACKGROUND

Little is known about long-term changes linking chronic diseases and poverty in low-income countries such as Bangladesh. This study examines how chronic disease mortality rates change across socioeconomic groups over time in Bangladesh, and whether such mortality is associated with households falling into poverty.

METHODS

Age-sex standardized chronic diseases mortality rates were estimated across socioeconomic groups in 1982, 1996 and 2005, using data from the health and demographic surveillance system in Matlab, Bangladesh. Changes in households falling below a poverty threshold after a chronic disease death were estimated between 1982-96 and 1996-2005.

RESULTS

Age-sex standardized chronic disease mortality rates rose from 646 per 100 000 population in 1982 to 670 in 2005. Mortality rates were higher in wealthier compared with poorer households in 1982 [Concentration Index = 0.037; 95% confidence interval (CI): 0.002, 0.072], but switched direction in 1996 (Concentration Index = -0.007; 95% CI: -0.023, 0.009), with an even higher concentration in the poor by 2005 (Concentration Index = -0.047; 95% CI: -0.061, -0.033). Between 1982-96 and 1996-2005, the highest chronic disease mortality rates were found among those households that fell below the poverty line. Households that had a chronic disease death in 1982 were 1.33 (95% CI: 1.03, 1.70) times more likely to fall below the poverty line in 1996 compared with households that did not.

CONCLUSIONS

Chronic disease mortality is a growing proportion of the disease burden in Bangladesh, with poorer households being more affected over time periods, leading to future household poverty.

Measuring causes of death in populations: a new metric that corrects cause-specific mortality fractions for chance

Background

Verbal autopsy is gaining increasing acceptance as a method for determining the underlying cause of death when the cause of death given on death certificates is unavailable or unreliable, and there are now a number of alternative approaches for mapping from verbal autopsy interviews to the underlying cause of death. For public health applications, the population-level aggregates of the underlying causes are of primary interest, expressed as the cause-specific mortality fractions (CSMFs) for a mutually exclusive, collectively exhaustive cause list. Until now, CSMF Accuracy is the primary metric that has been used for measuring the quality of CSMF estimation methods. Although it allows for relative comparisons of alternative methods, CSMF Accuracy provides misleading numbers in absolute terms, because even random allocation of underlying causes yields relatively high CSMF accuracy. Therefore, the objective of this study was to develop and test a measure of CSMF that corrects this problem.

Methods

We developed a baseline approach of random allocation and measured its performance analytically and through Monte Carlo simulation. We used this to develop a new metric of population-level estimation accuracy, the Chance Corrected CSMF Accuracy (CCCSMF Accuracy), which has value near zero for random guessing, and negative quality values for estimation methods that are worse than random at the population level.

Results

The CCCSMF Accuracy formula was found to be CCSMF Accuracy = (CSMF Accuracy - 0.632) / (1 - 0.632), which indicates that, at the population-level, some existing and commonly used VA methods perform worse than random guessing.

Conclusions

CCCSMF Accuracy should be used instead of CSMF Accuracy when assessing VA estimation methods because it provides a more easily interpreted measure of the quality of population-level estimates.

Electronic supplementary material

The online version of this article (doi:10.1186/s12963-015-0061-1) contains supplementary material, which is available to authorized users.

Global and regional burden of aortic dissection and aneurysms: mortality trends in 21 world regions, 1990 to 2010

A comprehensive and systematic assessment of the global burden of aortic aneurysms (AA) has been lacking. Therefore, we estimated AA regional deaths and years of life lost (YLL) in 21 regions worldwide for 1990 and 2010. We used the GBD (Global Burden of Disease) 2010 study causes of death database and the cause of death ensemble modeling approach to assess levels and trends of AA deaths by age, sex, and GBD region. The global AA death rate per 100,000 population was 2.49 (95% CI: 1.78 to 3.27) in 1990 and 2.78 (95% CI: 2.04 to 3.62) in 2010. In 1990 and 2010, the highest mean death rates were in Australasia and Western Europe: 8.82 (95% CI: 6.96 to 10.79) and 7.69 (95% CI: 6.11 to 9.57) in 1990 and 8.38 (95% CI: 6.48 to 10.86) and 7.68 (95% CI: 6.13 to 9.54) in 2010. YLL rates by GBD region mirrored the mortality rate pattern. Overall, men had higher AA death rates than women: 2.86 (95% CI: 1.90 to 4.22) versus 2.12 (95% CI: 1.33 to 3.00) in 1990 and 3.40 (95% CI: 2.26 to 5.01) versus 2.15 (95% CI: 1.44 to 2.89) in 2010. The relative change in median death rate was +0.22 (95% CI: 0.10 to 0.33) in developed nations versus +0.71 (95% CI: 0.28 to 1.40) in developing nations. The smallest relative changes in median death rate were noted in North America high income, Central Europe, Western Europe, and Australasia, with estimates of +0.07 (95% CI: -0.26 to 0.37), +0.08 (95% CI: -0.02 to 0.23), +0.09 (95% CI: -0.02 to 0.21), and +0.22 (95% CI: -0.08 to 0.46), respectively. The largest increases were in Asia Pacific high income, Southeast Asia, Latin America tropical, Oceania, South Asia, and Central Sub-Saharan Africa. Women rather than men drove the increase in the Asia Pacific high-income region: the relative change in median rates was +2.92 (95% CI: 0.6 to 4.35) versus +1.05 (95% CI: 0.61 to 2.42). In contrast to high-income regions, the observed pattern in developing regions suggests increasing AA burden, which portends future health system challenges in these regions.

Comparing verbal autopsy cause of death findings as determined by physician coding and probabilistic modelling: a public health analysis of 54 000 deaths in Africa and Asia

Background

Coverage of civil registration and vital statistics varies globally, with most deaths in Africa and Asia remaining either unregistered or registered without cause of death. One important constraint has been a lack of fit–for–purpose tools for registering deaths and assigning causes in situations where no doctor is involved. Verbal autopsy (interviewing care–givers and witnesses to deaths and interpreting their information into causes of death) is the only available solution. Automated interpretation of verbal autopsy data into cause of death information is essential for rapid, consistent and affordable processing.

Methods

Verbal autopsy archives covering 54 182 deaths from five African and Asian countries were sourced on the basis of their geographical, epidemiological and methodological diversity, with existing physician–coded causes of death attributed. These data were unified into the WHO 2012 verbal autopsy standard format, and processed using the InterVA–4 model. Cause–specific mortality fractions from InterVA–4 and physician codes were calculated for each of 60 WHO 2012 cause categories, by age group, sex and source. Results from the two approaches were assessed for concordance and ratios of fractions by cause category. As an alternative metric, the Wilcoxon matched–pairs signed ranks test with two one–sided tests for stochastic equivalence was used.

Findings

The overall concordance correlation coefficient between InterVA–4 and physician codes was 0.83 (95% CI 0.75 to 0.91) and this increased to 0.97 (95% CI 0.96 to 0.99) when HIV/AIDS and pulmonary TB deaths were combined into a single category. Over half (53%) of the cause category ratios between InterVA–4 and physician codes by source were not significantly different from unity at the 99% level, increasing to 62% by age group. Wilcoxon tests for stochastic equivalence also demonstrated equivalence.

Conclusions

These findings show strong concordance between InterVA–4 and physician–coded findings over this large and diverse data set. Although these analyses cannot prove that either approach constitutes absolute truth, there was high public health equivalence between the findings. Given the urgent need for adequate cause of death data from settings where deaths currently pass unregistered, and since the WHO 2012 verbal autopsy standard and InterVA–4 tools represent relatively simple, cheap and available methods for determining cause of death on a large scale, they should be used as current tools of choice to fill gaps in cause of death data.

Validating estimates of prevalence of non-communicable diseases based on household surveys: the symptomatic diagnosis study

BACKGROUND

Easy-to-collect epidemiological information is critical for the more accurate estimation of the prevalence and burden of different non-communicable diseases around the world. Current measurement is restricted by limitations in existing measurement systems in the developing world and the lack of biometry tests for non-communicable diseases. Diagnosis based on self-reported signs and symptoms ("Symptomatic Diagnosis," or SD) analyzed with computer-based algorithms may be a promising method for collecting timely and reliable information on non-communicable disease prevalence. The objective of this study was to develop and assess the performance of a symptom-based questionnaire to estimate prevalence of non-communicable diseases in low-resource areas.

METHODS

As part of the Population Health Metrics Research Consortium study, we collected 1,379 questionnaires in Mexico from individuals who suffered from a non-communicable disease that had been diagnosed with gold standard diagnostic criteria or individuals who did not suffer from any of the 10 target conditions. To make the diagnosis of non-communicable diseases, we selected the Tariff method, a technique developed for verbal autopsy cause of death calculation. We assessed the performance of this instrument and analytical techniques at the individual and population levels.

RESULTS

The questionnaire revealed that the information on health care experience retrieved achieved 66.1% (95% uncertainty interval [UI], 65.6-66.5%) chance corrected concordance with true diagnosis of non-communicable diseases using health care experience and 0.826 (95% UI, 0.818-0.834) accuracy in its ability to calculate fractions of different causes. SD is also capable of outperforming the current estimation techniques for conditions estimated by questionnaire-based methods.

CONCLUSIONS

SD is a viable method for producing estimates of the prevalence of non-communicable diseases in areas with low health information infrastructure. This technology can provide higher-resolution prevalence data, more flexible data collection, and potentially individual diagnoses for certain conditions.

Causes of death in two rural demographic surveillance sites in Bangladesh, 2004-2010: automated coding of verbal autopsies using InterVA-4

Objective

Population-based information on causes of death (CoD) by age, sex, and area is critical for countries with limited resources to identify and address key public health issues. This study analysed the demographic surveillance and verbal autopsy (VA) data to estimate age- and sex-specific mortality rates and cause-specific mortality fractions in two well-defined rural populations within the demographic surveillance system in Abhoynagar and Mirsarai subdistricts, located in different climatic zones.

Design

During 2004–2010, the sample demographic surveillance system registered 1,384 deaths in Abhoynagar and 1,847 deaths in Mirsarai. Trained interviewers interviewed the main caretaker of the deceased with standard VA questionnaires to record signs and symptoms of diseases or conditions that led to death and health care experiences before death. The computer-automated InterVA-4 method was used to analyse VAs to determine probable CoD.

Results

Age- and sex-specific death rates revealed a higher neonatal mortality rate in Abhoynagar than Mirsarai, and death rates and sex ratios of male to female death rates were higher in the ages after infancy. Communicable diseases (CDs) accounted for 16.7% of all deaths in Abhoynagar and 21.2% in Mirsarai – the difference was due mostly to more deaths from acute respiratory infections, pneumonia, and tuberculosis in Mirsarai. Non-communicable diseases (NCDs) accounted for 56.2 and 55.3% of deaths in each subdistrict, respectively, with leading causes being stroke (16.5–19.3%), neoplasms (13.2% each), cardiac diseases (8.9–11.6%), chronic obstructive pulmonary diseases (5.1–6.3%), diseases of the digestive system (3.1–4.1%), and diabetes (2.8–3.5%), together accounting for 49.2–51.2% points of the NCD deaths in the two subdistricts. Injury and other external causes accounted for another 7.5–7.7% deaths, with self-harm being higher among females in Abhoynagar.

Conclusions

The computer-automated coding of VA to determine CoD reconfirmed that NCDs were the leading CoD with some differences between the sites. Incorporating VA into the national sample vital registration system can help policy makers to identify the leading CoDs for public health planning.

Distribution of cause of death in rural Bangladesh during 2003-2010: evidence from two rural areas within Matlab Health and Demographic Surveillance site

Objective

This study used the InterVA-4 computerised model to assign probable cause of death (CoD) to verbal autopsies (VAs) generated from two rural areas, with a difference in health service provision, within the Matlab Health and Demographic Surveillance site (HDSS). This study aimed to compare CoD by gender, as well as discussing possible factors which could influence differences in the distribution of CoD between the two areas.

Design

Data for this study came from the Matlab the HDSS maintained by the International Centre for Diarrhoeal Disease Research, Bangladesh (icddr,b) since 1966. In late 1977, icddr,b divided HDSS and implemented a high-quality maternal, newborn and child health and family planning (MNCH-FP) services project in one half, called the icddr,b service area (SA), in addition to the usual public and private MNCH-FP services that serve the other half, called the government SA. HDSS field workers registered 12,144 deaths during 2003–2010, and trained interviewers obtained VA for 98.9% of them. The probabilistic model InterVA-4 probabilistic model (version 4.02) was used to derive probable CoD from VA symptoms. Cause-specific mortality rates and fractions were compared across gender and areas. Appropriate statistical tests were applied for significance testing.

Results

Mortality rates due to neonatal causes and communicable diseases (CDs) were lower in the icddr,b SA than in the government SA, where mortality rates due to non-communicable diseases (NCDs) were lower. Cause-specific mortality fractions (CSMFs) due to CDs (23.2% versus 18.8%) and neonatal causes (7.4% versus 6%) were higher in the government SA, whereas CSMFs due to NCDs were higher (58.2% versus 50.7%) in the icddr,b SA. The rank-order of CSMFs by age group showed marked variations, the largest category being acute respiratory infection/pneumonia in infancy, injury in 1–4 and 5–14 years, neoplasms in 15–49 and 50–64 years, and stroke in 65+ years.

Conclusions

Automated InterVA-4 coding of VA to determine probable CoD revealed the difference in the structure of CoD between areas with prominence of NCDs in both areas. Such information can help local planning of health services for prevention and management of disease burden.

Systematic review of statistics on causes of deaths in hospitals: strengthening the evidence for policy-makers

Objective

To systematically review the reliability of hospital data on cause of death and encourage periodic reviews of these data using a standard method.

Methods

We searched Google Scholar, Pubmed and Biblioteca Virtual de la Salud for articles in English, Spanish and Portuguese that reported validation studies of data on cause of death. We analysed the results of 199 studies that had used medical record reviews to validate the cause of death reported on death certificates or by the vital registration system.

Findings

The screened studies had been published between 1983 and 2013 and their results had been reported in English (n = 124), Portuguese (n = 25) or Spanish (n = 50). Only 29 of the studies met our inclusion criteria. Of these, 13 had examined cause of death patterns at the population level – with a view to correcting cause-specific mortality fractions – while the other 16 had been undertaken to identify discrepancies in the diagnosis for specific diseases before and after medical record review. Most of the selected studies reported substantial misdiagnosis of causes of death in hospitals. There was wide variation in study methodologies. Many studies did not describe the methods used in sufficient detail to be able to assess the reproducibility or comparability of their results.

Conclusion

The assumption that causes of death are being accurately reported in hospitals is unfounded. To improve the reliability and usefulness of reported causes of death, national governments should do periodic medical record reviews to validate the quality of their hospital cause of death data, using a standard.

A probabilistic method to estimate the burden of maternal morbidity in resource-poor settings: preliminary development and evaluation

Background

Maternal morbidity is more common than maternal death, and population-based estimates of the burden of maternal morbidity could provide important indicators for monitoring trends, priority setting and evaluating the health impact of interventions. Methods based on lay reporting of obstetric events have been shown to lack specificity and there is a need for new approaches to measure the population burden of maternal morbidity. A computer-based probabilistic tool was developed to estimate the likelihood of maternal morbidity and its causes based on self-reported symptoms and pregnancy/delivery experiences. Development involved the use of training datasets of signs, symptoms and causes of morbidity from 1734 facility-based deliveries in Benin and Burkina Faso, as well as expert review. Preliminary evaluation of the method compared the burden of maternal morbidity and specific causes from the probabilistic tool with clinical classifications of 489 recently-delivered women from Benin, Bangladesh and India.

Results

Using training datasets, it was possible to create a probabilistic tool that handled uncertainty of women’s self reports of pregnancy and delivery experiences in a unique way to estimate population-level burdens of maternal morbidity and specific causes that compared well with clinical classifications of the same data. When applied to test datasets, the method overestimated the burden of morbidity compared with clinical review, although possible conceptual and methodological reasons for this were identified.

Conclusion

The probabilistic method shows promise and may offer opportunities for standardised measurement of maternal morbidity that allows for the uncertainty of women’s self-reported symptoms in retrospective interviews. However, important discrepancies with clinical classifications were observed and the method requires further development, refinement and evaluation in a range of settings.

Performance of four computer-coded verbal autopsy methods for cause of death assignment compared with physician coding on 24,000 deaths in low- and middle-income countries

BACKGROUND

Physician-coded verbal autopsy (PCVA) is the most widely used method to determine causes of death (CODs) in countries where medical certification of death is uncommon. Computer-coded verbal autopsy (CCVA) methods have been proposed as a faster and cheaper alternative to PCVA, though they have not been widely compared to PCVA or to each other.

METHODS

We compared the performance of open-source random forest, open-source tariff method, InterVA-4, and the King-Lu method to PCVA on five datasets comprising over 24,000 verbal autopsies from low- and middle-income countries. Metrics to assess performance were positive predictive value and partial chance-corrected concordance at the individual level, and cause-specific mortality fraction accuracy and cause-specific mortality fraction error at the population level.

RESULTS

The positive predictive value for the most probable COD predicted by the four CCVA methods averaged about 43% to 44% across the datasets. The average positive predictive value improved for the top three most probable CODs, with greater improvements for open-source random forest (69%) and open-source tariff method (68%) than for InterVA-4 (62%). The average partial chance-corrected concordance for the most probable COD predicted by the open-source random forest, open-source tariff method and InterVA-4 were 41%, 40% and 41%, respectively, with better results for the top three most probable CODs. Performance generally improved with larger datasets. At the population level, the King-Lu method had the highest average cause-specific mortality fraction accuracy across all five datasets (91%), followed by InterVA-4 (72% across three datasets), open-source random forest (71%) and open-source tariff method (54%).

CONCLUSIONS

On an individual level, no single method was able to replicate the physician assignment of COD more than about half the time. At the population level, the King-Lu method was the best method to estimate cause-specific mortality fractions, though it does not assign individual CODs. Future testing should focus on combining different computer-coded verbal autopsy tools, paired with PCVA strengths. This includes using open-source tools applied to larger and varied datasets (especially those including a random sample of deaths drawn from the population), so as to establish the performance for age- and sex-specific CODs.