A screening method to spot biomarkers that may warn of serious events in a chronic disease - illustrated by cardiological CLARICOR trial data

OBJECTIVES

To develop a crude screening method for detecting biomarkers which frequently exhibit a rise (or fall) in level prior to a serious event (e.g. a stroke) in patients with a chronic disease, signalling that the biomarker may have an alarm-raising or prognostic potential. The subsequent assessment of the marker's clinical utility requires costly, difficult longitudinal studies. Therefore, initial screening of candidate-biomarkers is desirable.

METHODS

The method exploits a cohort of patients with biomarkers measured at entry and with recording of first serious event during follow-up. Copying those individual records onto a common timeline where a specific event occurs on the same day (Day 0) for all patients, the baseline biomarker level, when plotted against the patient's entry time on the revised timeline, will have a positive (negative) regression slope if biomarker levels generally rise (decline) the closer one gets to the event. As an example, we study 1,958 placebo-treated patients with stable coronary artery disease followed for nine years in the CLARICOR trial (NCT00121550), examining 11 newer biomarkers.

RESULTS

Rising average serum levels of cardiac troponin T and of N-terminal pro-B-type natriuretic peptide were seen prior to a fatal cardiovascular outcome. C-reactive protein rose prior to non-cardiovascular death. Glomerular filtration rate, seven lipoproteins, and nine newer cardiological biomarkers did not show convincing changes.

CONCLUSIONS

For early detection of biomarkers with an alarm-raising potential in chronic diseases, we proposed the described easy procedure. Using only baseline biomarker values and clinical course of participants with coronary heart disease, we identified the same cardiovascular biomarkers as those previously found containing prognostic information using longitudinal or survival analysis.

Trajectories in severe persistent pain after groin hernia repair: a retrospective analysis

Objectives

Severe persistent post-surgical pain (PPSP) remains a significant healthcare problem. In the third most common surgical procedure in the U.K., groin hernia repair, including 85,000 surgeries, estimated 1,500–3,000 patients will annually develop severe PPSP. While the trajectory of PPSP is generally considered a continuation of the acute post-surgery pain, recent data suggest the condition may develop with a delayed onset. This study evaluated pain-trajectories in a consecutive cohort referred from groin hernia repair-surgeons to a tertiary PPSP-center. Potential explanatory variables based on individual psychometric, sensory, and surgical profiles were analyzed.

Methods

Patients completed graphs on pain trajectories and questionnaires on neuropathic pain, pain-related functional assessments, and psychometrics. Surgical records and quantitative sensory testing profiles were obtained. Pain trajectories were normalized, and pre- and post-surgical segments were analyzed by a normalized area-under-the-curve (AUC) technique. Principal component analysis (PCA) was applied to the explanatory variables. Significant PCA-components were further examined using multiple logistic regression models.

Results

In 95 patients, the AUC identified groups of post-surgical pain trajectories (p<0.0001): group I (n=48), acute high-intensity pain progressing to PPSP; group II (n=28), delayed onset of PPSP; group III (n=7), repeat-surgery gradually inducing PPSP. Data from groups IV (n=3) and V (n=9) were not included in the statistical analysis due to small sample size and data heterogeneity, respectively. The PCA/logistic analyses indicated that neuropathic pain scores, composite pain scores, and pain-related functional assessments were explanatory variables for groups I and II.

Conclusions

Pain trajectories in PPSP after groin hernia repair are heterogeneous but can be classified into meaningful groups. Examination of pain trajectories, mirroring the transition from acute to severe persistent post-surgical pain, has the potential of uncovering clinically relevant pathophysiological mechanisms.

Prognostic value of 12 novel cardiological biomarkers in stable coronary artery disease. A 10-year follow-up of the placebo group of the Copenhagen CLARICOR trial

OBJECTIVE

To assess if 12 novel circulating biomarkers, when added to 'standard predictors' available in general practice, could improve the 10-year prediction of cardiovascular events and mortality in patients with stable coronary heart disease.

DESIGN

The patients participated as placebo receiving patients in the randomised clarithromycin for patients with stable coronary artery disease (CLARICOR) trial at a random time in their disease trajectory.

SETTING

Five Copenhagen University cardiology departments and a coordinating centre.

PARTICIPANTS

1998 participants with stable coronary artery disease.

OUTCOMES

Death and composite of myocardial infarction, unstable angina pectoris, cerebrovascular disease and death.

RESULTS

When only 'standard predictors' were included, 83.4% of all-cause death predictions and 68.4% of composite outcome predictions were correct. Log(calprotectin) and log(cathepsin-S) were not associated (p≥0.01) with the outcomes, not even as single predictors. Adding the remaining 10 biomarkers (high-sensitive assay cardiac troponin T; neutrophil gelatinase-associated lipocalin; osteoprotegerin; N-terminal pro-B-type natriuretic peptide; tumour necrosis factor receptor 1 and 2; pregnancy-associated plasma protein A; endostatin; YKL40; cathepsin-B), which were all individually significantly associated with the prediction of the two outcomes, increased the figures to 84.7% and 69.7%.

CONCLUSION

When 'standard predictors' routinely available in general practices are used for risk assessment in consecutively sampled patients with stable coronary artery disease, the addition of 10 novel biomarkers to the prediction model improved the correct prediction of all-cause death and the composite outcome by <1.5%.

TRIAL REGISTRATION NUMBER

NCT00121550.

Prognosis and Reclassification by YKL-40 in Stable Coronary Artery Disease

Background

The inflammatory biomarker YKL‐40 has previously been studied as a potential risk marker in cardiovascular disease. We aimed to assess the prognostic reclassification potential of serum YKL‐40 in patients with stable coronary artery disease.

Methods and Results

The main study population was the placebo group of the CLARICOR (Effect of Clarithromycin on Mortality and Morbidity in Patients With Ischemic Heart Disease) trial. The primary outcome was a composite of acute myocardial infarction, unstable angina pectoris, cerebrovascular disease, and all‐cause mortality. We used Cox proportional hazards regression models adjusted for C‐reactive protein level and baseline cardiovascular risk factors. Improvement in prediction by adding serum YKL‐40 to the risk factors was calculated using the Cox‐Breslow method and c‐statistic. A total of 2200 patients were randomized to placebo, with a follow‐up duration of 10 years. YKL‐40 was associated with an increased risk of the composite outcome (hazard ratio per unit increase in (YKL‐40) 1.13, 95% CI 1.03–1.24, P=0.013) and all‐cause mortality (hazard ratio 1.32, 95% CI 1.17–1.49, P<0.0001). Considering whether a composite‐outcome event was more likely to have, or not have, occurred to date, we found 68.4% of such predictions to be correct when based on the standard predictors, and 68.5% when serum YKL‐40 was added as a predictor. Equivalent results were obtained with c‐statistics.

Conclusions

Higher serum YKL‐40 was independently associated with an increased risk of adverse cardiovascular outcomes and mortality. Addition of YKL‐40 did not improve risk prediction in patients with stable coronary artery disease.

Clinical Trial Registration

URL: https://www.clinicaltrials.gov/. Unique identifier: NCT00121550.

Power estimations for non-primary outcomes in randomised clinical trials

OBJECTIVE AND METHODS

It is rare that trialists report power estimations of non-primary outcomes. In the present article, we will describe how to define a valid hierarchy of outcomes in a randomised clinical trial, to limit problems with Type I and Type II errors, using considerations on the clinical relevance of the outcomes and power estimations.

CONCLUSION

Power estimations of non-primary outcomes may guide trialists in classifying non-primary outcomes as secondary or exploratory. The power estimations are simple and if they are used systematically, more appropriate outcome hierarchies can be defined, and trial results will become more interpretable.

Cathepsin B and S as markers for cardiovascular risk and all-cause mortality in patients with stable coronary heart disease during 10 years: a CLARICOR trial sub-study

BACKGROUND AND AIMS

The lysosomal cysteine proteases cathepsin B and S have been implicated in the atherosclerotic process. The present paper investigates the association between serum levels of cathepsin B and S and cardiovascular events and mortality in patients with stable coronary heart disease.

METHODS

The CLARICOR trial is a randomised, placebo-controlled trial investigating the effect of clarithromycin versus placebo in patients with stable coronary heart disease. The outcome was time to either a cardiovascular event or all-cause mortality. The placebo group was used as discovery sample and the clarithromycin group as replication sample: n = 1998, n = 1979; mean age (years) 65, 65; 31%, 30% women; follow-up for 10 years; number of composite outcomes n = 1204, n = 1220; respectively. We used a pre-defined multivariable Cox regression model adjusting for inflammation, established cardiovascular risk factors, kidney function, and use of cardiovascular drugs.

RESULTS

Cathepsin B was associated with an increased risk of the composite outcome in both samples after multivariable adjustment (discovery: multivariable ratio (HR) per standard deviation increase 1.12, 95% confidence interval (CI) 1.05-1.19, p < 0.001, replication; HR 1.14, 95% CI 1.07-1.21, p < 0.001). There was no significant association between cathepsin S and the composite outcome in either the discovery or replication sample after multivariable adjustment (p>0.45). Secondary analyses suggest that cathepsin B was predominantly associated with mortality rather than specific cardiovascular events.

CONCLUSIONS

Cathepsin B, but not serum cathepsin S, was associated with an increased risk of cardiovascular events in patients with stable coronary heart disease. The clinical implications of our findings remain to be established.

Prognostic value of routinely available data in patients with stable coronary heart disease. A 10-year follow-up of patients sampled at random times during their disease course

Objective

To characterise the long-term prognosis of patients with stable coronary artery heart disease by means of 'standard predictors' defined as demographic, clinical and biochemical quantities routinely available in general practices and ascertained at an interview not prompted by renewed cardiac complaints.

Methods

This is an observational study based on data from 2199 Copenhagen placebo patients from the 'clarithromycin for patients with stable coronary heart disease' trial of patients with stable coronary heart disease. In the trial, we compared the effects of 14 days of clarithromycin treatment versus placebo. The predictors were based on the interview forms and blood samples collected at entry, along with demographic information from hospital files.We studied 'standard predictors' of a composite outcome (myocardial infarction, unstable angina, cerebrovascular disease or all-cause death) and of all-cause death. Using Cox regression, we compared predictions of status at 3, 6 and 9 years without and with the use of 'standard predictors' and used receiver operating characteristic statistic.

Results

Few 'standard predictors' were associated (p<0.01) with the composite outcome or with all-cause death. When no 'standard predictors' were included, 63.2% of the model-based predictions of the composite outcome and 79.9% of death predictions were correct. Including all 'standard predictors' in the model increased the figures to 68.4% and 83.4%, respectively. C indices were low, except when all-cause death was assessed as a single outcome where C was 0.79.

Conclusion

'Standard predictors' routinely available in general practices contribute only modestly to risk assessment in consecutively sampled patients with stable coronary heart disease as ascertained at a contact not prompted by renewed cardiac complaints. Novel biomarkers may improve the assessment.

Trial registration number

NCT00121550.

Minimum clinically important differences in chronic pain vary considerably by baseline pain and methodological factors: systematic review of empirical studies

BACKGROUND

The minimum clinically important difference (MCID) is used to interpret the relevance of treatment effects, e.g., when developing clinical guidelines, evaluating trial results or planning sample sizes. There is currently no agreement on an appropriate MCID in chronic pain and little is known about which contextual factors cause variation.

METHODS

This is a systematic review. We searched PubMed, EMBASE, and Cochrane Library. Eligible studies determined MCID for chronic pain based on a one-dimensional pain scale, a patient-reported transition scale of perceived improvement, and either a mean change analysis (mean difference in pain among minimally improved patients) or a threshold analysis (pain reduction associated with best sensitivity and specificity for identifying minimally improved patients). Main results were descriptively summarized due to considerable heterogeneity, which were quantified using meta-analyses and explored using subgroup analyses and metaregression.

RESULTS

We included 66 studies (31.254 patients). Median absolute MCID was 23 mm on a 0-100 mm scale (interquartile range [IQR] 12-39) and median relative MCID was 34% (IQR 22-45) among studies using the mean change approach. In both cases, heterogeneity was very high: absolute MCID I² = 99% and relative MCID I² = 96%. High variation was also seen among studies using the threshold approach: median absolute MCID was 20 mm (IQR 15-30) and relative MCID was 32% (IQR 15-41). Absolute MCID was strongly associated with baseline pain, explaining approximately two-thirds of the variation, and to a lesser degree with the operational definition of minimum pain relief and clinical condition. A total of 15 clinical and methodological factors were assessed as possible causes for variation in MCID.

CONCLUSIONS

MCID for chronic pain relief vary considerably. Baseline pain is strongly associated with absolute, but not relative, measures. To a much lesser degree, MCID is also influenced by the operational definition of relevant pain relief and possibly by clinical condition. Explicit and conscientious reflections on the choice of an MCID are required when classifying effect sizes as clinically important or trivial.

10-Year Associations Between Tumor Necrosis Factor Receptors 1 and 2 and Cardiovascular Events in Patients With Stable Coronary Heart Disease: A CLARICOR (Effect of Clarithromycin on Mortality and Morbidity in Patients With Ischemic Heart Disease) Trial Substudy

BACKGROUND

We aimed to assess the associations and predictive powers between the soluble receptors for tumor necrosis factor (TNF)-α (TNFR1 and TNFR2) and cardiovascular outcomes in patients with stable coronary heart disease.

METHODS AND RESULTS

CLARICOR (Effect of Clarithromycin on Mortality and Morbidity in Patients With Ischemic Heart Disease) is a randomized clinical trial comparing clarithromycin with placebo in patients with stable coronary heart disease. The primary outcome was a composite of nonfatal acute myocardial infarction, unstable angina pectoris, cerebrovascular disease, and all-cause mortality. Patients were followed up for 10 years; discovery sample, those assigned placebo (1204 events in n=1998); and replication sample, those assigned clarithromycin (1220 events in n=1979). We used Cox regression adjusted for C-reactive protein level, established cardiovascular risk factors, kidney function, and cardiovascular drugs. After adjustments, higher serum levels of TNFR1 and TNFR2 were associated with the composite outcome in the discovery sample (hazard ratio per SD increase, 1.13; 95% confidence interval, 1.05-1.22; P=0.001 for TNFR1; hazard ratio, 1.16; 95% confidence interval, 1.08-1.24; P<0.001 for TNFR2). The associations were similar in the replication sample. The associations with the composite outcome were mainly driven by acute myocardial infarction, cardiovascular mortality, and noncardiovascular mortality. The addition of TNFR1 and TNFR2 to established cardiovascular risk factors improved prediction only modestly (<1%).

CONCLUSIONS

Increased concentrations of circulating TNFR1 and TNFR2 were associated with increased risks of cardiovascular events and mortality in patients with stable coronary heart disease. Yet, the utility of measuring TNFR1 and TNFR2 to improve risk prediction in these patients appears limited.

CLINICAL TRIAL REGISTRATION

URL: https://www.clinicaltrials.gov. Unique identifier: NCT00121550.

The Measurement of Performance in Probabilistic Diagnosis

Owing to the inherent uncertainty of diagnostic tasks, diagnostic advice should be offered in a probabilistic, rather than deterministic form. Since the late fifties a lot of effort has been invested in constructing probabilistic diagnosis rules. Much less has been done to devise rational tools for evaluating them; conventional error rates reflect but one aspect of performance in a rather crude way. The aim of this paper and its successors is to offer a body of evaluation tools. After defining a general framework and stating its limitations we apply some graphical techniques to the acute abdominal pain data that will serve as illustration in the next papers as well: dot diagrams, the triangular diagram for the three-disease case, and three tabular representations based 011 categorization of the probabilities, viz. the usual (forced) classification matrix, from which various classification rates are read off; the classification matrix with doubt, in which uncertain and confident diagnoses are distinguished; and the exclusion matrix, which spots diseases that are judged improbable. Together these matrices give a good first impression of the behaviour of a probabilistic diagnosis system. The outlined techniques of later papers are needed for a more complete analysis.

The Measurement of Performance in Probabilistic Diagnosis

Owing to the inherent uncertainty of diagnostic tasks, diagnostic advice should be offered in a probabilistic, rather than deterministic form. Since the late fifties a lot of effort has been invested in constructing probabilistic diagnosis rules. Much less has been done to devise rational tools for evaluating them ; conventional error rates reflect but one aspect of performance in a rather crude way. The aim of this paper and its successors is to offer a body of evaluation tools. After defining a general framework and stating its limitations we apply some graphical techniques to the acute abdominal pain data that will serve as illustration in the next papers as well: dot diagrams, the triangular diagram for the three-disease case, and three tabular representations based on categorization of the probabilities, viz. the usual (forced) classification matrix, from which various classification rates are read off; the classification matrix with doubt, in which uncertain and confident diagnoses are distinguished; and the exclusion matrix, which spots diseases that are judged improbable. Together these matrices give a good first impression of the behaviour of a probabilistic diagnosis system. The outlined techniques of later papers are needed for a more complete analysis.

The Measurement of Performance in Probabilistic Diagnosis V. General Recommendations

Guidelines are offered concerning the use of evaluation tools, with emphasis on the simplest and most informative ones amongst those proposed in Parts I—IV of this series. Moreover, the main areas of future research and development, as the authors see them, are pointed out.

The Measurement of Performance in Probabilistic Diagnosis IV. Utility Considerations in Therapeutics and Prognostics

It is argued that it is preferable to evaluate probabilistic diagnosis systems in terms of utility (patient benefit) or loss (negative benefit). We have adopted the provisional strategy of scoring performance as if the system were the actual decision-maker (not just an aid to him) and argue that a rational figure of merit is given by the average loss which patients would incur by having the system decide on treatment, the treatment being selected according to the minimum expected loss principle of decision theory.

A similar approach is taken to the problem of evaluating probabilistic prognoses, but the fundamental differences between treatment selection skill and prognostic skill and their implications for the assessment of such skills are stressed. The necessary elements of decision theory are explained by means of simple examples mainly taken from the acute abdomen, and the proposed evaluation tools are applied to Acute Abdominal Pain data analysed in our previous papers by other (not decision-theoretic) means. The main difficulty of the decision theory approach, viz. that of obtaining good medical utility values upon which the analysis can be based, receives due attention, and the evaluation approach is extended to cover more realistic situations in which utility or loss values vary from patient to patient.

Pain relief that matters to patients: systematic review of empirical studies assessing the minimum clinically important difference in acute pain

BACKGROUND

The minimum clinically important difference (MCID) is used to interpret the clinical relevance of results reported by trials and meta-analyses as well as to plan sample sizes in new studies. However, there is a lack of consensus about the size of MCID in acute pain, which is a core symptom affecting patients across many clinical conditions.

METHODS

We identified and systematically reviewed empirical studies of MCID in acute pain. We searched PubMed, EMBASE and Cochrane Library, and included prospective studies determining MCID using a patient-reported anchor and a one-dimensional pain scale (e.g. 100 mm visual analogue scale). We summarised results and explored reasons for heterogeneity applying meta-regression, subgroup analyses and individual patient data meta-analyses.

RESULTS

We included 37 studies (8479 patients). Thirty-five studies used a mean change approach, i.e. MCID was assessed as the mean difference in pain score among patients who reported a minimum degree of improvement, while seven studies used a threshold approach, i.e. MCID was assessed as the threshold in pain reduction associated with the best accuracy (sensitivity and specificity) for identifying improved patients. Meta-analyses found considerable heterogeneity between studies (absolute MCID: I² = 93%, relative MCID: I² = 75%) and results were therefore presented qualitatively, while analyses focused on exploring reasons for heterogeneity. The reported absolute MCID values ranged widely from 8 to 40 mm (standardised to a 100 mm scale) and the relative MCID values from 13% to 85%. From analyses of individual patient data (seven studies, 918 patients), we found baseline pain strongly associated with absolute, but not relative, MCID as patients with higher baseline pain needed larger pain reduction to perceive relief. Subgroup analyses showed that the definition of improved patients (one or several categories improvement or meaningful change) and the design of studies (single or multiple measurements) also influenced MCID values.

CONCLUSIONS

The MCID in acute pain varied greatly between studies and was influenced by baseline pain, definitions of improved patients and study design. MCID is context-specific and potentially misguiding if determined, applied or interpreted inappropriately. Explicit and conscientious reflections on the choice of a reference value are required when using MCID to classify research results as clinically important or trivial.

Predictors for major cardiovascular outcomes in stable ischaemic heart disease (PREMAC): statistical analysis plan for data originating from the CLARICOR (clarithromycin for patients with stable coronary heart disease) trial

BACKGROUND

The purpose of the predictors for major cardiovascular outcomes in stable ischaemic heart disease (PREMAC) study is exploratory and hypothesis generating. We want to identify biochemical quantities which-conditionally on the values of available standard demographic, anamnestic, and biochemical data-may improve the prediction of cardiovascular outcomes and/or death in patients suffering from stable ischaemic heart disease. The candidate biochemical quantities include N-terminal pro-B-type natriuretic peptide, YKL-40, osteoprotegerin, high-sensitive assay cardiac troponin T (hs-cTnT), pregnancy-associated plasma protein-A (PAPP-A), cathepsin B, cathepsin S, soluble TNF receptor 1 and 2, neutrophil gelatinase-associated lipocalin, endostatin, and calprotectin. As an extra objective, we also want to assess if skewness in these predictors may explain why the clarithromycin for patients with stable coronary heart disease (CLARICOR) trial found increased all-cause and cardiovascular (CV) mortality on a brief clarithromycin regimen compared with placebo.

METHODS

Baseline data were obtained from the hospital files at five cardiology clinics covering the Copenhagen area. The CLARICOR trial included data from 4372 stable coronary artery disease patients recruited among such patients alive and diagnosed with acute myocardial infarction or unstable angina pectoris during 1993 to 1999 in Copenhagen and randomised during October 1999 to April 2000 to the CLARICOR trial of 14 days clarithromycin versus placebo.Initial follow-up lasted for 2.6 years, during which outcomes were collected through hospital and death registries and assessed by an adjudication committee. Corresponding register data later showed to produce similar results. The adjudicated outcomes were therefore replaced and augmented by register data on outcomes to cover 10 years of follow-up. Biochemical marker data were obtained from analysis of serum from the CLARICOR bio-bank collected at randomisation and stored at -80° C.Using Cox proportional hazard method, we will identify among the candidate biochemical quantities those which are significant predictors when used alone and in combination with the standard predictors as defined in the present study.

DISCUSSION

Patients who became stable during the period 1993 to 1999 and died before October 1999 are missing. The data from the placebo patients are nevertheless useful to identify new prognostic biomarkers in patients with stable coronary artery disease, and data from both trial groups are useful to assess important potential skewness between randomised groups. However, due to the potential selection bias, we do not feel that it is advisable to try to rank identified biochemical predictors relative to each other nor to use the results for predictive purposes.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT00121550 Date of registration 13 July 2005Date of enrolment of first participant 12 October 1999.

Risk of low birthweight in social districts of Copenhagen

The purpose of this survey was to investigate the small-area variations in low birthweight within social services districts in Copenhagen and the relation of such variations to the socioeconomic characteristics of the district. The study was based on register data and included all live-born singleton births to women living in the city of Copenhagen from 1987-90. We found a statistically significant association between district and risk of newborns being small-for-gestational age (SGA). This association was independent of adjustment for maternal age and parity. Only part of the association was explained, though this was still significant, using an index summarizing the socioeconomic living conditions in the district. Splitting up the index into its four constituent components, however, showed that the association between SGA and district was primarily due to housing conditions in the districts. Mapping the districts demonstrated a clear tendency of clustering of the high-risk areas with poor socioeconomic conditions.

Early-Phase Studies of Biomarkers: What Target Sensitivity and Specificity Values Might Confer Clinical Utility?

BACKGROUND

Many cancer biomarker research studies seek to develop markers that can accurately detect or predict future onset of disease. To design and evaluate these studies, one must specify the levels of accuracy sought. However, justified target levels are rarely available.

METHODS

We describe a way to calculate target levels of sensitivity and specificity for a biomarker intended to be applied in a defined clinical context. The calculation requires knowledge of the prevalence or incidence of cases in the clinical population and the ratio of benefit associated with the clinical consequences of a positive biomarker test in cases (true positive) to cost associated with a positive biomarker test in controls (false positive). Guidance is offered on soliciting the cost/benefit ratio. The calculations are based on the longstanding decision theory concept of providing a net benefit on average in the population, and they rely on some assumptions about uniformity of costs and benefits to those tested.

RESULTS

Calculations are illustrated with 3 applications: predicting colon cancer recurrence in stage 1 patients; predicting interval breast cancer (between mammography screenings); and screening for ovarian cancer.

CONCLUSIONS

It is feasible to specify target levels of biomarker performance that enable evaluation of the potential clinical impact of biomarkers in early-phase studies. Nevertheless, biomarkers meeting the criteria should still be tested rigorously in studies that measure the actual impact on patient outcomes of using the biomarker to make clinical decisions.

The matching quality of experimental and control interventions in blinded pharmacological randomised clinical trials: a methodological systematic review

Background

Blinding is a pivotal method to avoid bias in randomised clinical trials. In blinded drug trials, experimental and control interventions are often designed to be matched, i.e. to appear indistinguishable. It is unknown how often matching procedures are inadequate, so we decided to systematically identify and analyse studies of matching quality in drug trials. Our primary objective was to assess the proportion of studies that concluded that the matching was inadequate; our secondary objective was to describe mechanisms for inadequate matching.

Methods

Systematic review. We searched PubMed, Google Scholar and Web of Science Citation Index for studies that assessed whether supposedly indistinguishable interventions (experimental and control) in randomized clinical drug trials could be distinguished based on physical properties (e.g. appearance or smell). Two persons decided on study eligibility and extracted data independently. Our primary analysis was based on the conclusions of each study. In supportive analyses, we defined a low and a high threshold for inadequate matching. We summarised results qualitatively.

Results

We included studies of 36 trials, of which 28 (78 %) were published before 1977. The studies differed considerably with regard to design, methodology and analysis. Sixteen of the 36 studies (44 %) concluded inadequate matching. When we adapted high or low thresholds for inadequate matching, the number of trials with inadequate matching was reduced to 12 (33 %) or increased to 26 (72 %). Inadequate matching was concluded in 7 of 22 trials (32 %) based on a defined cohort of trials. Inadequate matching was concluded in 9 of 14 trials (64 %) which were not based on a trial cohort, and therefore at a higher risk of publication bias. The proportion of inadequate matching did not seem to depend on publication year. Typical mechanisms of inadequate matching were differences in taste or colour.

Conclusion

We identified matching quality studies of 36 randomized clinical drug trials. Sixteen of the 36 studies (44 %) concluded inadequate matching. Few studies of matching quality in contemporary trials have been published, but show similar results as found for older trials. Inadequate matching in drug trials may be more prevalent than commonly believed.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-016-0111-9) contains supplementary material, which is available to authorized users.

Clarithromycin for stable coronary heart disease increases all-cause and cardiovascular mortality and cerebrovascular morbidity over 10years in the CLARICOR randomised, blinded clinical trial

BACKGROUND

The CLARICOR trial reported that clarithromycin compared with placebo increased all-cause mortality in patients with stable coronary heart disease. This study investigates the effects of clarithromycin versus placebo during 10years follow up.

METHODS

The CLARICOR trial is a randomised, placebo-controlled trial including 4373 patients with stable coronary heart disease. The interventions were 2weeks of clarithromycin 500mg a day versus placebo. 10year follow up was performed through Danish public registers and analysed with Cox regression.

RESULTS

Clarithromycin increased all-cause mortality (hazard ratio (HR): 1.10, 95% confidence interval (CI): 1.00-1.21) and cerebrovascular disease during 10years (HR: 1.19, 95% CI: 1.02-1.38). The increased mortality and morbidity were restricted to patients not on statin at entry (HR: 1.16, 95% CI: 1.04-1.31, and HR: 1.25, 95% CI: 1.03-1.50). The assumption of constant HR during the 10years was violated for cardiovascular death (P=0.01) and cardiovascular death outside hospital (P<0.0005). Analyses of the effects over time showed that clarithromycin increased cardiovascular mortality during the first three years (HR: 1.42, 95% CI: 1.09-1.84) due to increased cardiovascular mortality outside hospital in patients not on statin (HR: 2.36, 95% CI: 1.60-3.50). During the last 4years, cardiovascular death outside hospital was lower in the clarithromycin group (HR: 0.64, 95% CI: 0.46-0.88).

CONCLUSION

Clarithromycin increased mortality due to cardiovascular death outside hospital and cerebrovascular morbidity in patients with stable coronary heart disease who were not on statin. The increased cardiovascular mortality was years later compensated, likely through frailty attrition.

Agreement between public register and adjudication committee outcome in a cardiovascular randomized clinical trial

The objective of this study is to describe the agreement between randomized trial outcome assessment by committee and outcomes entirely identified through public registers.

METHODS

In the CLARICOR trial, 4,372 patients with stable coronary heart disease received a short course of clarithromycin versus placebo and were followed up for 2.6 years. The pertinent hospital records and death certificates had originally been evaluated by the adjudication committee using common definitions of outcomes mapped into a 6-category list. We now mechanically converted the International Classification of Diseases-coded diagnoses of the public registries into the same categories. After cross-tabulation of the committee diagnoses with National Patient Register diagnoses and Register of Causes of Death, we calculate agreement and compare the estimated intervention effects of the 2 data sets.

RESULTS

With public register data, the protocol-specified categories were slightly more frequent. Overall agreement was 74% for hospital discharges and 60% for cause of death, but the intervention effect, expressed as a hazard ratio, stayed within 4% of the value originally obtained with the adjudication committee (P ≥ .35).

CONCLUSIONS

Our results show a modest agreement between formal adjudication and outcomes deducible from public registers. However, the estimated intervention effect did not differ noticeably between the 2 data sources. If studies on a wide range of public registers confirm these findings, register outcomes may be considered as a replacement for adjudication committees.

Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies

BACKGROUND

Blinding patients in clinical trials is a key methodological procedure, but the expected degree of bias due to nonblinded patients on estimated treatment effects is unknown.

METHODS

Systematic review of randomized clinical trials with one sub-study (i.e. experimental vs control) involving blinded patients and another, otherwise identical, sub-study involving nonblinded patients. Within each trial, we compared the difference in effect sizes (i.e. standardized mean differences) between the sub-studies. A difference <0 indicates that nonblinded patients generated a more optimistic effect estimate. We pooled the differences with random-effects inverse variance meta-analysis, and explored reasons for heterogeneity.

RESULTS

Our main analysis included 12 trials (3869 patients). The average difference in effect size for patient-reported outcomes was -0.56 (95% confidence interval -0.71 to -0.41), (I(2)=60%, P=0.004), i.e. nonblinded patients exaggerated the effect size by an average of 0.56 standard deviation, but with considerable variation. Two of the 12 trials also used observer-reported outcomes, showing no indication of exaggerated effects due lack of patient blinding. There was a larger effect size difference in 10 acupuncture trials [-0.63 (-0.77 to -0.49)], than in the two non-acupuncture trials [-0.17 (-0.41 to 0.07)]. Lack of patient blinding also increased attrition and use of co-interventions: ratio of control group attrition risk 1.79 (1.18 to 2.70), and ratio of control group co-intervention risk 1.55 (0.99 to 2.43).

CONCLUSIONS

This study provides empirical evidence of pronounced bias due to lack of patient blinding in complementary/alternative randomized clinical trials with patient-reported outcomes.

Observer bias in randomized clinical trials with time-to-event outcomes: systematic review of trials with both blinded and non-blinded outcome assessors

BACKGROUND

We wanted to evaluate the impact of nonblinded outcome assessors on estimated treatment effects in time-to-event trials.

METHODS

Systematic review of randomized clinical trials with both blinded and nonblinded assessors of the same time-to-event outcome. Two authors agreed on inclusion of trials and outcomes. We compared hazard ratios based on nonblinded and blinded assessments. A ratio of hazard ratios (RHR)<1 indicated that nonblinded assessors generated more optimistic effect estimates. We pooled RHRs with inverse variance random-effects meta-analysis.

RESULTS

We included 18 trials. Eleven trials (1969 patients) with subjective outcomes provided hazard ratios, RHR 0.88 (0.69 to 1.12), (I2=44%, P=0.06), but unconditional pooling was problematic because of qualitative heterogeneity. Four atypical cytomegalovirus retinitis trials compared experimental oral administration with control intravenous administration of the same drug, resulting in bias favouring the control intervention, RHR 1.33 (0.98 to 1.82). Seven trials of cytomegalovirus retinitis, tibial fracture and multiple sclerosis compared experimental interventions with standard control interventions, e.g. placebo, no-treatment or active control, resulting in bias favouring the experimental intervention, RHR 0.73 (0.57 to 0.93), indicating an average exaggeration of nonblinded hazard ratios by 27% (7% to 43%).

CONCLUSIONS

Lack of blinded outcome assessors in randomized trials with subjective time-to-event outcomes causes high risk of observer bias. Nonblinded outcome assessors typically favour the experimental intervention, exaggerating the hazard ratio by an average of approximately 27%; but in special situations, nonblinded outcome assessors favour control interventions, inducing a comparable degree of observer bias in the reversed direction.

Risk stratification in stable coronary artery disease is possible at cardiac troponin levels below conventional detection and is improved by use of N-terminal pro-B-type natriuretic peptide

AIMS

Low prevalence of detectable cardiac troponin in healthy people and low-risk patients previously curtailed its use. With a new high-sensitive cardiac troponin assay (hs-cTnT), concentrations below conventional detection may have prognostic value, notably in combination with N-terminal pro-B-type natriuretic peptide (NT-pro-BNP).

METHODS AND RESULTS

Biomarker concentrations were determined from serum obtained at enrolment in the CLARICOR trial involving 4197 patients with stable coronary artery disease (CAD) followed for 2.6 years. Serum hs-cTnT was detectable (above 3 ng/l) in 78% and above the conventional 99th percentile (13.5 ng/l) in 23%. Across all levels of hs-cTnT there was a graded increase in the risk of cardiovascular death after adjustment for known prognostic indicators: hazard ratio (HR) per unit increase in the natural logarithm of the hs-cTnT level, 1.49; 95% confidence interval (CI), 1.23-1.81; similarly for all-cause mortality (HR 1.48, 95% CI 1.29-1.70) and myocardial infarction (HR 1.37, 95% CI 1.13-1.67). Increasing values of hs-cTnT were associated with increased mortality across all values of NT-pro-BNP, but this was particularly prominent when NT-pro-BNP >400 ng/l.

CONCLUSIONS

In patients with stable CAD, any detectable hs-cTnT level is significantly associated with all-cause mortality, cardiovascular death, and myocardial infarction after adjustment for traditional risk factors and NT-pro-BNP. Excess mortality is particularly pronounced in patients with NT-pro-BNP >400 ng/l.

Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors

BACKGROUND

Clinical trials are commonly done without blinded outcome assessors despite the risk of bias. We wanted to evaluate the effect of nonblinded outcome assessment on estimated effects in randomized clinical trials with outcomes that involved subjective measurement scales.

METHODS

We conducted a systematic review of randomized clinical trials with both blinded and nonblinded assessment of the same measurement scale outcome. We searched PubMed, EMBASE, PsycINFO, CINAHL, Cochrane Central Register of Controlled Trials, HighWire Press and Google Scholar for relevant studies. Two investigators agreed on the inclusion of trials and the outcome scale. For each trial, we calculated the difference in effect size (i.e., standardized mean difference between nonblinded and blinded assessments). A difference in effect size of less than 0 suggested that nonblinded assessors generated more optimistic estimates of effect. We pooled the differences in effect size using inverse variance random-effects meta-analysis and used metaregression to identify potential reasons for variation.

RESULTS

We included 24 trials in our review. The main meta-analysis included 16 trials (involving 2854 patients) with subjective outcomes. The estimated treatment effect was more beneficial when based on nonblinded assessors (pooled difference in effect size -0.23 [95% confidence interval (CI) -0.40 to -0.06]). In relative terms, nonblinded assessors exaggerated the pooled effect size by 68% (95% CI 14% to 230%). Heterogeneity was moderate (I(2) = 46%, p = 0.02) and unexplained by metaregression.

INTERPRETATION

We provide empirical evidence for observer bias in randomized clinical trials with subjective measurement scale outcomes. A failure to blind assessors of outcomes in such trials results in a high risk of substantial bias.

Serum YKL-40 predicts long-term mortality in patients with stable coronary disease: a prognostic study within the CLARICOR trial

OBJECTIVE

We investigated whether the inflammatory biomarker YKL-40 could improve the long-term prediction of death made by common risk factors plus high-sensitivity C-reactive protein (hs-CRP) and N-terminal-pro-B natriuretic peptide (NT-proBNP) in patients with stable coronary artery disease (CAD).

BACKGROUND

Non-hospitalized CAD patients are usually followed in general practice. There is a need for identify biomarkers which could help to foresee the prognoses of these patients. Elevated serum YKL-40 is a short-term predictor for myocardial infarction, cardiovascular mortality and all-cause mortality in patients with stable CAD.

METHODS

Serum YKL-40, hs-CRP, and NT-proBNP were measured in 4265 (97.6%) of the 4372 patients with stable CAD included in the CLARICOR trial, and death was registered in a 6-years follow-up period.

RESULTS

The median serum YKL-40 was 110 μg/L [IQR=93], hs-CRP 2.8 mg/L [IQR=4.74], and NT-proBNP 203 ng/L [IQR=407]. During 6 years follow-up period 923 (21.1%) patients died. After adjustment for type of intervention, risk factors (age, sex, hypertension, diabetes, smoking status, and previous myocardial infarction) and medical treatment (diuretics, digoxin, and statin) serum YKL-40 (transformed as ln(max(82, YKL-40/μg/L)) was significantly associated with all-cause mortality [hazard ratio (HR)=1.55, 95% CI=1.39-1.73, p<0.001]. After additional adjustment for ln(hs-CRP) and ln(NT-proBNP) this was still true [HR=1.38, 95% CI=1.21-1.53, p<0.001].

CONCLUSIONS

Serum YKL-40 is a predictor of long-term mortality in patients with stable CAD independent of common risk factors and ln(hs-CRP) and ln(NT-proBNP). Serum YKL-40 can be used for prognostication in these patients.

Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors

OBJECTIVE

To evaluate the impact of non-blinded outcome assessment on estimated treatment effects in randomised clinical trials with binary outcomes.

DESIGN

Systematic review of trials with both blinded and non-blinded assessment of the same binary outcome. For each trial we calculated the ratio of the odds ratios--the odds ratio from non-blinded assessments relative to the corresponding odds ratio from blinded assessments. A ratio of odds ratios <1 indicated that non-blinded assessors generated more optimistic effect estimates than blinded assessors. We pooled the individual ratios of odds ratios with inverse variance random effects meta-analysis and explored reasons for variation in ratios of odds ratios with meta-regression. We also analysed rates of agreement between blinded and non-blinded assessors and calculated the number of patients needed to be reclassified to neutralise any bias.

DATA SOURCES

PubMed, Embase, PsycINFO, CINAHL, Cochrane Central Register of Controlled Trials, HighWire Press, and Google Scholar.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Randomised clinical trials with blinded and non-blinded assessment of the same binary outcome.

RESULTS

We included 21 trials in the main analysis (with 4391 patients); eight trials provided individual patient data. Outcomes in most trials were subjective--for example, qualitative assessment of the patient's function. The ratio of the odds ratios ranged from 0.02 to 14.4. The pooled ratio of odds ratios was 0.64 (95% confidence interval 0.43 to 0.96), indicating an average exaggeration of the non-blinded odds ratio by 36%. We found no significant association between low ratios of odds ratios and scores for outcome subjectivity (P=0.27); non-blinded assessor's overall involvement in the trial (P=0.60); or outcome vulnerability to non-blinded patients (P=0.52). Blinded and non-blinded assessors agreed in a median of 78% of assessments (interquartile range 64-90%) in the 12 trials with available data. The exaggeration of treatment effects associated with non-blinded assessors was induced by the misclassification of a median of 3% of the assessed patients per trial (1-7%).

CONCLUSIONS

On average, non-blinded assessors of subjective binary outcomes generated substantially biased effect estimates in randomised clinical trials, exaggerating odds ratios by 36%. This bias was compatible with a high rate of agreement between blinded and non-blinded outcome assessors and driven by the misclassification of few patients.

Chlamydia pneumoniae IgG and IgA antibody titers and prognosis in patients with coronary heart disease: results from the CLARICOR trial

The association observed between coronary heart disease (CHD) and Chlamydia (Chlamydophila) pneumoniae antibodies prompted, during the 1990s, several primary and secondary prevention trials with various antibiotics. In our CLARICOR trial, a randomized placebo-controlled trial in 4372 patients with stable CHD, a brief clarithromycin regimen was followed, unexpectedly, by increased long-term mortality. We now compare C. pneumoniae antibody levels at entry with population levels, with the patients' individual histories, and with their subsequent outcomes. IgG antibody levels were somewhat raised, but elevated IgA and IgG titers were unrelated to entry data (including prior acute myocardial infarction), except for an association with smoking and with not using statins. Hazards of mortality and of other outcomes tended to slightly increase with IgA and decrease with IgG titers, but the unfavorable clarithromycin effect was unrelated to antibody levels and remains unexplained. Smoking-related lung disease probably underlies the link between heart disease and increased IgG titers.

Simultaneous use of serum IgG and IgM for risk scoring of suspected early Lyme borreliosis: graphical and bivariate analyses

The laboratory diagnosis of early disseminated Lyme borreliosis (LB) rests on IgM and IgG antibodies in serum. The purpose of this study was to refine the statistical interpretation of IgM and IgG by combining the diagnostic evidence provided by the two immunoglobulins and exploiting the whole range of the quantitative variation in test values. ELISA assays based on purified flagella antigen were performed on sera from 815 healthy Danish blood donors as negative controls and 117 consecutive patients with confirmed neuroborreliosis (NB). A logistic regression model combining the standardized units of the IgM and IgG ELISA assays was constructed and the resulting disease risks graphically evaluated by receiver operating characteristic and 'predictiveness' curves. The combined model improves the discrimination between NB patients and blood donors. Hence, it is possible to report a predicted risk of disease graded for each individual patient, as is theoretically preferable. The predictiveness curve, when adapted to the local pretest probability of LB, allows high-risk and low-risk thresholds to be defined instead of cut-offs based on the laboratory characteristics only, and it allows the extent of under- and over-treatment to be assessed. It is shown that an example patient with low ELISA results in IgM and IgG, considered negative by the conventional cut-off, has a relatively high risk of belonging to the truly diseased population and a low risk of being false positive. Using a 20% high-risk threshold for advising the clinician to consider treatment, the sensitivity of the assay is increased from 76% to 85%, while the specificity is maintained at around 95%.

The Osteoporosis Self-Assessment Tool versus alternative tests for selecting postmenopausal women for bone mineral density assessment: a comparative systematic review of accuracy

SUMMARY

We performed a systematic review of studies comparing the Osteoporosis Self-Assessment Tool (OST) and other tests used to select women for bone mineral density (BMD) assessment. In comparative meta-analyses, we found that the accuracy of OST was similar to other tests that are based on information from the medical history. By contrast, assessment by quantitative ultrasonography at the heel was more accurate than OST in discriminating between women with high and low BMD. The methodological quality of the included studies was generally low.

INTRODUCTION

Numerous tests are suggested for triaging postmenopausal women for bone mineral density (BMD) assessment by dual-energy X-ray absorptiometry. Previous studies suggest that OST, based on age and weight only, may be as accurate as more complex triage tests. We systematically compare the accuracy of OST and alternative triage tests in postmenopausal women.

METHODS

We searched PubMed, Embase, Web of Science, citation lists, and conference proceedings. Our main measure of accuracy was the diagnostic odds ratio (DOR). We compared summary estimates of DOR (sDOR) for OST and alternative tests in pairwise meta-analyses by using the Moses-Littenberg approach.

RESULTS

Summary estimates of DOR for OST and the clinical decision rules Simple Calculated Osteoporosis Risk Estimation (SCORE) and Osteoporosis Risk Assessment Instrument (ORAI) did not differ significantly in white women (relative sDOR: 0.57-1.17, all p >or= 0.11). By contrast, sDOR was higher for Stiffness Index assessed by calcaneal quantitative ultrasonography than for OST (relative sDOR: 1.9, p = 0.005). Studies were few in Asian and black women. Methodological quality, assessed with the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) checklist, was generally low.

CONCLUSIONS

In white women, the accuracy of OST and alternative clinical decision rules was similar, whereas Stiffness Index was more accurate than OST. Low study quality renders transferability to clinical settings uncertain.

[The effect of length of stay and number of relocations on asylum-seeking children's mental health--secondary publication]

Among asylum-seeking children aged 4-16 years living in the asylum centres managed by the Danish Red Cross in 2006, we investigated mental health in relation to length of stay and number of relocations. Mental health was evaluated using the Strengths and Difficulties Questionnaire (SDQ). We found that children who had been asylum-seeking for more than one year had an increased risk of having mental difficulties (odds ratio 5.5; 95% confidence interval 1.8-16.3); four or more relocations in the asylum system were also associated with a higher risk of mental problems (odds ratio 3,0; 1,4-6,7).

Mental health among children seeking asylum in Denmark--the effect of length of stay and number of relocations: a cross-sectional study

BACKGROUND

The process of seeking asylum and the related organisational conditions in the host country may adversely affect the children's mental health. The objective of this study was to examine the mental health of children seeking asylum in relation to organisational factors of the asylum system including length of stay and number of relocations.

METHODS

The population included all 260 parent-accompanied asylum-seeking children aged 4-16 years living in the asylum centres managed by the Danish Red Cross in October-December 2006. Mental health was evaluated using the Strengths and Difficulties Questionnaire. School teachers evaluated children aged 4-16; and the 11-16-year-olds completed the self-report version. To assess the association between organisational factors and mental health, binary logistic regression analyses were done using backwards elimination. We received responses for 246 children equivalent to 95% of the study population.

RESULTS

Using teachers' reports, we found that children who had been asylum-seeking for more than one year in Denmark had an increased risk of having mental difficulties (odds ratio 5.5, 95% CI 1.8-16.3); four or more relocations in the asylum system were also associated with a higher risk (3.0, 1.4-6.7). When the self-report data were included, the associations were even stronger.

CONCLUSION

Protracted stays at asylum centres and multiple relocations within the asylum system appear to have an adverse effect on asylum-seeking children's mental health. A limit to the duration of the children's stay in the asylum system should be ensured. Follow-up studies with inclusion of other conditions, such as parental mental health and the children's previous trauma, are needed to clarify the influence of the different factors and their interactions.

Clarithromycin for 2 weeks for stable coronary heart disease: 6-year follow-up of the CLARICOR randomized trial and updated meta-analysis of antibiotics for coronary heart disease

OBJECTIVES

We have reported increased 2.6-year mortality in clarithromycin- versus placebo-exposed stable coronary heart disease patients, but meta-analysis of randomized trials in coronary heart disease patients showed no significant effect of antibiotics on mortality. Here we report the 6-year mortality of clarithromycin- versus placebo-exposed patients and updated meta-analyses.

METHODS

Centrally randomized, placebo controlled multicenter trial. All parties were blinded. Analyses were by intention to treat. Meta-analyses followed the Cochrane Collaboration methodology.

RESULTS

We randomized 4,372 patients with stable coronary heart disease to clarithromycin 500 mg (n = 2,172) or placebo (n = 2,200) once daily for 2 weeks. Mortality was followed through public register. Nine hundred and twenty-three patients (21.1%) died. Six-year mortality was significantly higher in the clarithromycin group (hazard ratio 1.21, 95% confidence interval 1.06-1.38). Adjustment for entry characteristics (sex, age, prior myocardial infarction, center, and smoking) did not change the results (1.18, 1.04-1.35). Addition of our data to that of other randomized trials on antibiotics for patients with coronary heart disease versus placebo/no intervention (17 trials, 25,271 patients, 1,877 deaths) showed a significantly increased relative risk of death from antibiotics of 1.10 (1.01-1.20) without heterogeneity.

CONCLUSIONS

Our results stress the necessity to consider carefully the strength of the indication before administering antibiotics to patients with coronary heart disease.