Modelling risk when binary outcomes are subject to error

Bayesian mixed hidden Markov models: a multi-level approach to modeling categorical outcomes with differential misclassification

Questionnaire-based health status outcomes are often prone to misclassification. When studying the effect of risk factors on such outcomes, ignoring any potential misclassification may lead to biased effect estimates. Analytical challenges posed by these misclassified outcomes are further complicated when simultaneously exploring factors for both the misclassification and health processes in a multi-level setting. To address these challenges, we propose a fully Bayesian mixed hidden Markov model (BMHMM) for handling differential misclassification in categorical outcomes in a multi-level setting. The BMHMM generalizes the traditional hidden Markov model (HMM) by introducing random effects into three sets of HMM parameters for joint estimation of the prevalence, transition, and misclassification probabilities. This formulation not only allows joint estimation of all three sets of parameters but also accounts for cluster-level heterogeneity based on a multi-level model structure. Using this novel approach, both the true health status prevalence and the transition probabilities between the health states during follow-up are modeled as functions of covariates. The observed, possibly misclassified, health states are related to the true, but unobserved, health states and covariates. Results from simulation studies are presented to validate the estimation procedure, to show the computational efficiency due to the Bayesian approach and also to illustrate the gains from the proposed method compared to existing methods that ignore outcome misclassification and cluster-level heterogeneity. We apply the proposed method to examine the risk factors for both asthma transition and misclassification in the Southern California Children's Health Study.

Bias--is it a problem, and what should we do?

Observational studies are prone to two types of errors: random and systematic. Random error arises as a result of variation between samples that might be drawn in a study and can be reduced by increasing the sample size. Systematic error arises from problems with the study design or the methods used to obtain the study data and is not influenced by sample size. Over the last 20 years, veterinary epidemiologists have made great progress in dealing more effectively with random error (particularly through the use of multilevel models) but paid relatively little attention to systematic error. Systematic errors can arise from unmeasured confounders, selection bias and information bias. Unmeasured confounders include both factors which are known to be confounders but which were not measured in a study and factors which are not known to be confounders. Confounders can bias results toward or away from the null. The impact of selection bias can also be difficult to predict and can be negligible or large. Although the direction of information bias is generally toward the null, this cannot be guaranteed and its impact might be very large. Methods of dealing with systematic errors include: qualitative assessment, quantitative bias analysis and incorporation of bias parameters into the statistical analyses.

A Bayesian model for misclassified binary outcomes and correlated survival data with applications to breast cancer

Breast cancer patients may experience ipsilateral breast tumor relapse (IBTR) after breast conservation therapy. IBTR is classified as either true local recurrence or new ipsilateral primary tumor. The correct classification of IBTR status has significant implications in therapeutic decision-making and patient management. However, the diagnostic tests to classify IBTR are imperfect and prone to misclassification. In addition, some observed survival data (e.g., time to relapse, time from relapse to death) are strongly correlated with IBTR status. We present a Bayesian approach to model the potentially misclassified IBTR status and the correlated survival information. We conduct the inference using a Bayesian framework via Markov chain Monte Carlo simulation implemented in WinBUGS. Extensive simulation shows that the proposed method corrects biases and provides more efficient estimates for the covariate effects on the probability of IBTR and the diagnostic test accuracy. Moreover, our method provides useful subject-specific patient prognostic information. Our method is motivated by, and applied to, a dataset of 397 breast cancer patients.

Correction for misclassification of caries experience in the absence of internal validation data

OBJECTIVES

To quantify the effects of risk factors and/or determinants on disease occurrence, it is important that the risk factors as well as the variable that measures the disease outcome are recorded with the least error as possible. When investigating the factors that influence a binary outcome, a logistic regression model is often fitted under the assumption that the data are collected without error. However, most categorical outcomes (e.g., caries experience) are accompanied by misclassification and this needs to be accounted for. The aim of this research was to adjust for binary outcome misclassification using an external validation study when investigating factors influencing caries experience in schoolchildren.

MATERIALS AND METHODS

Data from the Signal Tandmobiel(®) study were used. A total of 500 children from the main and 148 from the validation study were included in the analysis. Regression models (with several covariates) for sensitivity and specificity were used to adjust for misclassification in the main data.

RESULTS

The use of sensitivity and specificity modeled as functions of several covariates resulted in a better correction compared to using point estimates of sensitivity and specificity. Age, geographical location of the school to which the child belongs, dentition type, tooth type, and surface type were significantly associated with the prevalence of caries experience.

CONCLUSIONS

Sensitivity and specificity calculated based on an external validation study may resemble those obtained from an internal study if conditioned on a rich set of covariates.

CLINICAL RELEVANCE

Main data can be corrected for misclassification using information obtained from an external validation study when a rich set of covariates is recorded during calibration.

Epidemiology of coagulase-negative staphylococci intramammary infection in dairy cattle and the effect of bacteriological culture misclassification

Objectives of this study were to identify the manageable risk factors associated with the lactational incidence, elimination, and prevalence of coagulase-negative staphylococci (CNS) intramammary infections (IMI) while taking into account the difficulties inherent to their diagnosis. A second objective was to evaluate the effect of CNS IMI misclassification in mastitis research. A cohort of 90 Canadian dairy herds was followed throughout 2007 to 2008. In each herd, series of quarter milk samples were collected from a subsample of cows and bacteriological culture was performed to identify prevalent, incident, and eliminated CNS IMI. Practices used on farms were captured using direct observations and a validated questionnaire. The relationships between herd CNS IMI prevalence and herd incidence and elimination rates were explored using linear regression. Manageable risk factors associated with the prevalence, incidence, or elimination of CNS IMI were identified via Bayesian analyses using a latent class model approach, allowing adjustment of the estimates for the imperfect sensitivity and specificity of bacteriological culture. After adjustment for the diagnostic test limitations, a mean CNS IMI quarter prevalence of 42.7% [95% confidence interval (CI): 34.7, 50.1] and incidence and elimination rates of 0.29 new IMI/quarter-month (95% CI: 0.21, 0.37) and 0.79 eliminated IMI/quarter-month (95% CI: 0.66, 0.91), respectively, were observed. Considerable biases of the estimates were observed when CNS IMI misclassification was ignored. These biases were important for measures of association with risk factors, were almost always toward the null value, and led to both type I and type II errors. Coagulase-negative staphylococci IMI incidence appeared to be a stronger determinant of herd IMI prevalence than IMI elimination rate. The majority of herds followed were already using blanket dry cow treatment and postmilking teat disinfection. A holistic approach considering associations with all 3 outcomes was used to interpret associations between manageable risk factors and CNS IMI. Sand and wood-based product bedding showed desirable associations with CNS IMI compared with straw bedding. Quarters of cows that had access to pasture during the sampling period had lower odds of acquiring a new CNS IMI and of having a prevalent CNS IMI. Many practices showed an association with only one of the CNS outcomes and should, therefore, be considered with caution.

Sample size estimation to substantiate freedom from disease for clustered binary data with a specific risk profile

Disease cases are often clustered within herds or generally groups that share common characteristics. Sample size formulae must adjust for the within-cluster correlation of the primary sampling units. Traditionally, the intra-cluster correlation coefficient (ICC), which is an average measure of the data heterogeneity, has been used to modify formulae for individual sample size estimation. However, subgroups of animals sharing common characteristics, may exhibit excessively less or more heterogeneity. Hence, sample size estimates based on the ICC may not achieve the desired precision and power when applied to these groups. We propose the use of the variance partition coefficient (VPC), which measures the clustering of infection/disease for individuals with a common risk profile. Sample size estimates are obtained separately for those groups that exhibit markedly different heterogeneity, thus, optimizing resource allocation. A VPC-based predictive simulation method for sample size estimation to substantiate freedom from disease is presented. To illustrate the benefits of the proposed approach we give two examples with the analysis of data from a risk factor study on Mycobacterium avium subsp. paratuberculosis infection, in Danish dairy cattle and a study on critical control points for Salmonella cross-contamination of pork, in Greek slaughterhouses.

Validation data-based adjustments for outcome misclassification in logistic regression: an illustration

Misclassification of binary outcome variables is a known source of potentially serious bias when estimating adjusted odds ratios. Although researchers have described frequentist and Bayesian methods for dealing with the problem, these methods have seldom fully bridged the gap between statistical research and epidemiologic practice. In particular, there have been few real-world applications of readily grasped and computationally accessible methods that make direct use of internal validation data to adjust for differential outcome misclassification in logistic regression. In this paper, we illustrate likelihood-based methods for this purpose that can be implemented using standard statistical software. Using main study and internal validation data from the HIV Epidemiology Research Study, we demonstrate how misclassification rates can depend on the values of subject-specific covariates, and we illustrate the importance of accounting for this dependence. Simulation studies confirm the effectiveness of the maximum likelihood approach. We emphasize clear exposition of the likelihood function itself, to permit the reader to easily assimilate appended computer code that facilitates sensitivity analyses as well as the efficient handling of main/external and main/internal validation-study data. These methods are readily applicable under random cross-sectional sampling, and we discuss the extent to which the main/internal analysis remains appropriate under outcome-dependent (case-control) sampling.

Self-report versus ultrasound measurement of uterine fibroid status

BACKGROUND

Much of the epidemiologic research on risk factors for fibroids, the leading indication for hysterectomy, relies on self-reported outcome. Self-report is subject to misclassification because many women with fibroids are undiagnosed. The purpose of this analysis was to quantify the extent of misclassification and identify associated factors.

METHODS

Self-reported fibroid status was compared to ultrasound screening from 2046 women in Right From The Start (RFTS) and 869 women in the Uterine Fibroid Study (UFS). Log-binomial regression was used to estimate sensitivity (Se) and specificity (Sp) and examine differences by ethnicity, age, education, body mass index, parity, and miscarriage history.

RESULTS

Overall sensitivity was ≤0.50. Sensitivity was higher in blacks than whites (RFTS: 0.34 vs. 0.23; UFS: 0.58 vs. 0.32) and increased with age. Parous women had higher sensitivity than nulliparae, especially in RFTS whites (Se ratio=2.90; 95% confidence interval [CI]: 1.51, 5.60). Specificity was 0.98 in RFTS and 0.86 in UFS. Modest ethnic differences were seen in UFS (Sp ratio, black vs. white=0.90; 95% CI: 0.81, 0.99). Parity was inversely associated with specificity, especially among UFS black women (Sp ratio=0.84; 95% CI: 0.73, 0.97). Among women who reported a previous diagnosis, a shorter time interval between diagnosis and ultrasound was associated with increased agreement between the two measures.

CONCLUSIONS

Misclassification of fibroid status can differ by factors of etiologic interest. These findings are useful for assessing (and correcting) bias in studies using self-reported clinical diagnosis as the outcome measure.

Modeling continuous diagnostic test data using approximate Dirichlet process distributions

There is now a large literature on the analysis of diagnostic test data. In the absence of a gold standard test, latent class analysis is most often used to estimate the prevalence of the condition of interest and the properties of the diagnostic tests. When test results are measured on a continuous scale, both parametric and nonparametric models have been proposed. Parametric methods such as the commonly used bi-normal model may not fit the data well; nonparametric methods developed to date have been relatively complex to apply in practice, and their properties have not been carefully evaluated in the diagnostic testing context. In this paper, we propose a simple yet flexible Bayesian nonparametric model which approximates a Dirichlet process for continuous data. We compare results from the nonparametric model with those from the bi-normal model via simulations, investigating both how much is lost in using a nonparametric model when the bi-normal model is correct and how much can be gained in using a nonparametric model when normality does not hold. We also carefully investigate the trade-offs that occur between flexibility and identifiability of the model as different Dirichlet process prior distributions are used. Motivated by an application to tuberculosis clustering, we extend our nonparametric model to accommodate two additional dichotomous tests and proceed to analyze these data using both the continuous test alone as well as all three tests together.

Classification of ipsilateral breast tumor recurrences after breast conservation therapy can predict patient prognosis and facilitate treatment planning

OBJECTIVE

To classify ipsilateral breast tumor recurrences (IBTR) as either new primary tumors (NP) or true local recurrence (TR). We utilized 2 different methods and compared sensitivities and specificities between them. Our goal was to determine whether distinguishing NP from TR had prognostic value.

BACKGROUND

After breast-conservation therapy, IBTR may be classified into 2 distinct types (NP and TR). Studies have attempted to classify IBTR by using tumor location, histologic subtype, DNA flow cytometry data, or gene-expression profiling data.

METHODS

A total of 447 (7.9%) of 5660 patients undergoing breast-conservation therapy from 1970 to 2005 experienced IBTR. Clinical data from 397 patients were available for review. We classified IBTRs as NP or TR on the basis of either tumor location and histologic subtype (method 1) or tumor location, histologic subtype, estrogen receptor status and human epidermal growth factor receptor 2 status (method 2). Kaplan-Meier curves and log-rank tests were used to evaluate overall and disease-specific survival differences between the 2 groups. Classification methods were validated by calculating sensitivity and specificity values using a Bayesian method.

RESULTS

Of 397 patients, 196 (49.4%) were classified as NP by method 1 and 212 (53.4%) were classified as NP by method 2. The sensitivity and specificity values were 0.812 and 0.867 for method 1 and 0.870 and 0.800 for method 2, respectively. Regardless of method used, patients classified as NP developed contralateral breast carcinoma more often but had better 10-year overall and disease-specific survival rates than those classified as TR. Patients with TR were more likely to develop metastatic disease after IBTR.

CONCLUSION

Ipsilateral breast tumor recurrences classified as TR and NP had clinically different features, suggesting that classifying IBTR may provide clinically significant data for the management of IBTR.

Meta-regression with partial information on summary trial or patient characteristics

We present a model for meta-regression in the presence of missing information on some of the study level covariates, obtaining inferences using Bayesian methods. In practice, when confronted with missing covariate data in a meta-regression, it is common to carry out a complete case or available case analysis. We propose to use the full observed data, modelling the joint density as a factorization of a meta-regression model and a conditional factorization of the density for the covariates. With the inclusion of several covariates, inter-relations between these covariates are modelled. Under this joint likelihood-based approach, it is shown that the lesser assumption of the covariates being Missing At Random is imposed, instead of the more usual Missing Completely At Random (MCAR) assumption. The model is easily programmable in WinBUGS, and we examine, through the analysis of two real data sets, sensitivity and robustness of results to the MCAR assumption.

Hepatitis C infection among injecting drug users in England and Wales (1992-2006): there and back again?

Changes in hepatitis C virus (HCV) prevalence from 1992 to 2006 were examined by using 24,311 records from unlinked anonymous surveillance of injecting drug users in England and Wales. Bayesian logistic regression was used to estimate annual prevalence, accounting for changing recruitment patterns (age, gender, injecting duration, geographic region, interactions) and the sensitivity and specificity of different oral fluid testing devices. After controlling for these differences, the authors found that the adjusted HCV prevalence decreased from 70% (95% credible interval: 62, 78) in 1992 to 47% (95% credible interval: 43, 51) in 1998 before rising again to 53% (95% credible interval: 48, 58) in 2006. Women injecting drug users had a higher HCV risk than did men (odds ratio = 1.50, 95% credible interval: 1.31, 1.73). Two regions (London and North West) had a markedly higher HCV prevalence than did the rest of England and Wales. Among individuals who had injected for less than 1 year, the adjusted HCV prevalence in 2006 was higher than that in 1992 (28% vs. 19%, respectively). HCV infection can be prevented. The public health challenge in England and Wales is to increase action in order to regain a downward trend in HCV risk and the benefit that has been lost since 1998.

Bayesian approach to average power calculations for binary regression models with misclassified outcomes

We develop a simulation-based procedure for determining the required sample size in binomial regression risk assessment studies when response data are subject to misclassification. A Bayesian average power criterion is used to determine a sample size that provides high probability, averaged over the distribution of potential future data sets, of correctly establishing the direction of association between predictor variables and the probability of event occurrence. The method is broadly applicable to any parametric binomial regression model including, but not limited to, the popular logistic, probit, and complementary log-log models. We detail a common medical scenario wherein ascertainment of true disease status is impractical or otherwise impeded, and in its place the outcome of a single binary diagnostic test is used as a surrogate. These methods are then extended to the two diagnostic test setting. We illustrate the method with categorical covariates using one example that involves screening for human papillomavirus. This example coupled with results from simulated data highlights the utility of our Bayesian sample size procedure with error prone measurements.

A Bayesian approach to adjust for diagnostic misclassification between two mortality causes in Poisson regression

Response misclassification of counted data biases and understates the uncertainty of parameter estimators in Poisson regression models. To correct these problems, researchers have devised classical procedures that rely on asymptotic distribution results and supplemental validation data in order to estimate unknown misclassification parameters. We derive a new Bayesian Poisson regression procedure that accounts and corrects for misclassification for a count variable with two categories. Under the Bayesian paradigm, one can use validation data, expert opinion, or a combination of these two approaches to correct for the consequences of misclassification. The Bayesian procedure proposed here yields an operationally effective way to correct and account for misclassification effects in Poisson count regression models. We demonstrate the performance of the model in a simulation study. Additionally, we analyze two real-data examples and compare our new Bayesian inference method that adjusts for misclassification with a similar analysis that ignores misclassification.

Binary regression with misclassified response and covariate subject to measurement error: a bayesian approach

We consider a Bayesian analysis for modeling a binary response that is subject to misclassification. Additionally, an explanatory variable is assumed to be unobservable, but measurements are available on its surrogate. A binary regression model is developed to incorporate the measurement error in the covariate as well as the misclassification in the response. Unlike existing methods, no model parameters need be assumed known. Markov chain Monte Carlo methods are utilized to perform the necessary computations. The methods developed are illustrated using atomic bomb survival data. A simulation experiment explores advantages of the approach.

Robustness of prevalence estimates derived from misclassified data from administrative databases

Because primary data collection can be expensive, researchers are increasingly using information collected in medical administrative databases for scientific purposes. This information, however, is typically collected for reasons other than research, and many such databases have been shown to contain substantial proportions of misclassification errors. For example, many administrative databases contain fields for patient diagnostic codes, but these are often missing or inaccurate, in part because physician reimbursement schemes depend on medical acts performed rather than any diagnosis. Errors in ascertaining which individuals have a given disease bias not only prevalence estimates, but also estimates of associations between the disease and other variables, such as medication use. We attempt to estimate the prevalence of osteoarthritis (OA) among elderly Quebeckers using a government administrative database. We compare a naive estimate relying solely on the physician diagnoses of OA listed in the database to estimates from several different Bayesian latent class models which adjust for misclassified physician diagnostic codes via use of other available diagnostic clues. We find that the prevalence estimates vary widely, depending on the model used and assumptions made. We conclude that any inferences from these databases need to be interpreted with great caution, until further work estimating the reliability of database items is carried out.

Correction of nutrition test errors for more accurate quantification of the link between dental health and malnutrition

OBJECTIVE

We quantified the link between tooth deterioration and malnutrition in institutionalized elderly subjects, taking into account the major risk factors for malnutrition and adjusting for the measurement error made in using the Mini Nutritional Assessment questionnaire.

METHODS

Data stem from a survey conducted in 2005 in 1094 subjects >or=60 y of age from a large sample of 100 institutions of the Rhône-Alpes region of France. A Bayesian approach was used to quantify the effect of tooth deterioration on malnutrition through a two-level logistic regression. This approach allowed taking into account the uncertainty on sensitivity and specificity of the Mini Nutritional Assessment questionnaire to adjust for the measurement error of that test.

RESULTS

After adjustment for other risk factors, the risk of malnutrition increased significantly and continuously 1.15 times (odds ratio 1.15, 95% credibility interval 1.06-1.25) whenever the masticatory percentage decreased by 10 points, which is equivalent to the loss of two molars. The strongest factors that augmented the probability of malnutrition were deglutition disorders, depression, and verbal inconsistency. Dependency was also an important factor; the odds of malnutrition nearly doubled for each additional grade of dependency (graded 6 to 1). Diabetes, central neurodegenerative disease, and carcinoma tended to increase the probability of malnutrition but their effect was not statistically significant.

CONCLUSION

Dental status should be considered a serious risk factor for malnutrition. Regular dental examination and care should preserve functional dental integrity to prevent malnutrition in institutionalized elderly people.

Bayesian statistics in medicine: a 25 year review

This review examines the state of Bayesian thinking as Statistics in Medicine was launched in 1982, reflecting particularly on its applicability and uses in medical research. It then looks at each subsequent five-year epoch, with a focus on papers appearing in Statistics in Medicine, putting these in the context of major developments in Bayesian thinking and computation with reference to important books, landmark meetings and seminal papers. It charts the growth of Bayesian statistics as it is applied to medicine and makes predictions for the future. From sparse beginnings, where Bayesian statistics was barely mentioned, Bayesian statistics has now permeated all the major areas of medical statistics, including clinical trials, epidemiology, meta-analyses and evidence synthesis, spatial modelling, longitudinal modelling, survival modelling, molecular genetics and decision-making in respect of new technologies.

A guide for multilevel modeling of dyadic data with binary outcomes using SAS PROC NLMIXED

In the social and health sciences, data are often structured hierarchically, with individuals nested within groups. Dyads constitute a special case of hierarchically structured data with variation at both the individual and dyadic level. Analyses of data from dyads pose several challenges due to the interdependence between members within dyads and issues related to small group sizes. Multilevel analytic techniques have been developed and applied to dyadic data in an attempt to resolve these issues. In this article, we describe a set of analyses for modeling individual- and dyad-level influences on binary outcomes using SAS statistical software; and we discuss the benefits and limitations of such an approach. For illustrative purposes, we apply these techniques to estimate individual-dyad-level predictors of viral hepatitis C infection among heterosexual couples in East Harlem, New York City.