Modeling temporal relationships in large scale clinical associations

OBJECTIVE

We describe an approach for modeling temporal relationships in a large scale association analysis of electronic health record data. The addition of temporal information can inform hypothesis generation and help to explain the relationships. We applied this approach on a dataset containing 41.2 million time-stamped International Classification of Diseases, Ninth Revision (ICD-9) codes from 1.6 million patients.

METHODS

We performed two independent analyses including a pairwise association analysis using a χ(2) test and a temporal analysis using a binomial test. Data were visualized using network diagrams and reviewed for clinical significance.

RESULTS

We found nearly 400 000 highly associated pairs of ICD-9 codes with varying numbers of strong temporal associations ranging from ≥1 day to ≥10 years apart. Most of the findings were not considered clinically novel, although some, such as an association between Helicobacter pylori infection and diabetes, have recently been reported in the literature. The temporal analysis in our large cohort, however, revealed that diabetes usually preceded the diagnoses of H pylori, raising questions about possible cause and effect.

DISCUSSION

Such analyses have significant limitations, some of which are due to known problems with ICD-9 codes and others to potentially incomplete data even at a health system level. Nevertheless, large scale association analyses with temporal modeling can help provide a mechanism for novel discovery in support of hypothesis generation.

CONCLUSIONS

Temporal relationships can provide an additional layer of meaning in identifying and interpreting clinical associations.

Combining the regression discontinuity design and propensity score-based weighting to improve causal inference in program evaluation

The regression discontinuity (RD) design is considered to be the closest to a randomized trial that can be applied in non-experimental settings. The design relies on a cut-off point on a continuous baseline variable to assign individuals to treatment. The individuals just to the right and left of the cut-off are assumed to be exchangeable - as in a randomized trial. Any observed discontinuity in the relationship between the assignment variable and outcome is therefore considered evidence of a treatment effect. In this paper, we describe key advances in the RD design over the past decade and illustrate their implementation using data from a health management intervention. We then introduce the propensity score-based weighting technique as a complement to the RD design to correct for imbalances in baseline characteristics between treated and non-treated groups that may bias RD results. We find that the weighting strategy outperforms standard regression covariate adjustment in the present data. One clear advantage of the weighting technique over regression covariate adjustment is that we can directly inspect the degree to which balance was achieved. Because of its relative simplicity and tremendous utility, the RD design (either alone or combined with propensity score weighting adjustment) should be considered as an alternative approach to evaluate health management program effectiveness when using observational data.

Reclassification of ICD-9 Codes into Meaningful Categories for Oncology Survivorship Research

Background. The International Classification of Disease, ninth revision (ICD-9) is designed to code disease into categories which are placed into administrative databases. These databases have been used for epidemiological studies. However, the categories used in the ICD9-codes are not always the most effective for evaluating specific diseases or their outcomes, such as the outcomes of cancer treatment. Therefore a re-classification of the ICD-9 codes into new categories specific to cancer outcomes is needed. Methods. An expert panel comprised of two physicians created broad categories that would be most useful to researchers investigating outcomes and morbidities associated with the treatment of cancer. A Senior Data Coordinator with expertise in ICD-9 coding, then joined this panel and each code was re-classified into the new categories. Results. Consensus was achieved for the categories to go from the 17 categories in ICD-9 to 39 categories. The ICD-9 Codes were placed into new categories, and subcategories were also created for more specific outcomes. The results of this re-classification is available in tabular form. Conclusions. ICD-9 codes were re-classified by group consensus into categories that are designed for oncology survivorship research. The novel re-classification system can be used by those involved in cancer survivorship research.

How to measure the outcomes of chronic disease management

The fastest-growing methodology for disease management outcomes measurement is valid, transparent, easy to apply, and freely available in the public domain and this article. It measures the actual goal of disease management, which is to reduce the rate of adverse events associated with the disease(s) being managed. Changes in this rate can be translated into a return on investment using some explicit assumptions about comorbidities and episode costs. Outcomes measured in this way show that in the health plan community as a whole, disease management in the broadest sense is working, as measured by the relative stability in the rate of adverse medical events closely associated with common chronic disease during this decade of increasing prevalence of most of the common chronic conditions.

The relationship between chronic and non-chronic trends

Cost trend plays a crucial role in evaluating the outcome of disease management (DM) programs, which typically attempt to manage members who have any of 5 major chronic conditions (the chronic population) through targeted intervention. A widely adopted methodology for evaluating DM outcomes is commonly referred to as the "pre-post" method, also known as the adjusted historical control method. One drawback of the pre-post approach is the need for a valid trend adjustment so that program year and baseline year costs can be compared to determine the impact of the DM program. The trend adjustment plays a crucial role in the pre-post methodology because the estimated cost savings is very sensitive to the variation in the trend estimate. DMAA: The Care Continuum Alliance recommends that the concurrent trend for members who do not have any of the 5 chronic conditions (the non-chronic population) be used to estimate the chronic trend. One major assumption in using this method is that there is a relatively stable relationship between the chronic trend and the non-chronic trend for the same population in the absence of the DM program. In this analysis, the relationship between the two trends is studied based on empirical data. Some key factors that may impact the relationship between the chronic and non-chronic trends are also examined. Medical and pharmacy claims from a large commercial client over a 4-year period were collected for the analysis. This client did not have any robust DM program in place during the 4-year period. The major findings from this study include: (1) The method used to identify the chronic population for calculating the trend has a significant impact on the relationship between the chronic and non-chronic trends. With the Annual Qualification method, in which the chronic members are identified using the claims in a specific time period regardless of whether they have been identified previously or thereafter, the relationship between the non-chronic and chronic trends appears stable over time. With the Prospective method, the non-chronic trend does not serve as a good estimate of the chronic trend. (2) It is important to use risk adjustment when calculating trends to account for risk changes within the population from year to year. The chronic and non-chronic trends converge further when risk adjustment is applied. Based on the empirical evidence from the data used in this study, the concurrent non-chronic trend could be used to help estimate the chronic trend if the proper member selection method (Annual Qualification) is used to select the chronic population and proper steps are taken to adjust the different risk levels from year to year. One should not conclude, however, that such a stable relationship between chronic and non-chronic trend has been universally validated based on this single study. This study highlights the importance of carefully validating trend relationships from a particular population before assuming a stable trend relationship in DM program outcome evaluation.