Progress of therapy in acute leukemia 1948-1981: randomized versus nonrandomized clinical trials

Shortcomings in the clinical evaluation of new drugs: acute myeloid leukemia as paradigm

Drugs introduced over the past 25 years have benefitted many patients with acute myeloid leukemia (AML) and provided cure for some. Still, AML remains difficult to treat, and most patients will eventually die from their disease. Therefore, novel drugs and drug combinations are under intense investigation, and promising results eagerly awaited and embraced. However, drug development is lengthy and costs are staggering. While the phase 1-phase 2-phase 3 sequence of clinical drug testing has remained inviolate for decades, it appears intrinsically inefficient, and scientific flaws have been noted by many authors. Of major concern is the high frequency of false-positive results obtained in phase 2 studies. Here, we review features of phase 2 trials in AML that may contribute to this problem, particularly lack of control groups, patient heterogeneity, selection bias, and choice of end points. Recognizing these problems and challenges should provide us with opportunities to make drug development more efficient and less costly. We also suggest strategies for trial design improvement. Although our focus is on the treatment of AML, the principles that we highlight should be broadly applicable to the evaluation of new treatments for a variety of diseases.

Impact of random assignment on study outcome: an empirical examination

Sixty research investigations published in the biomedical literature were analyzed to examine the effect of design attributes on outcome. All 60 studies included a controlled trial involving a pretest, a therapeutic intervention, and a posttest across at least two groups. Thirty of the trials used random assignment of participants to treatment or control conditions and 30 trials employed some nonrandom method of subject assignment. Trial results were aggregated and evaluated by comparing effect sizes for the primary statistical test of the hypothesis. Data analysis revealed that the trial results, as measured by effect size, did not vary across therapeutic trials using random assignment versus those using nonrandom allocation of subjects. The impact of design attributes in the interpretation of multiple clinical trials addressing a similar research question is examined. The argument is made that various design attributes frequently associated with methodological quality should be considered as important moderator variables and their influence on trial outcome should not be assumed a priori but rather examined empirically.

How study design affects outcomes in comparisons of therapy. I: Medical

We analysed 113 reports published in 1980 in a sample of medical journals to relate features of study design to the magnitude of gains attributed to new therapies over old. Overall we rated 87 per cent of new therapies as improvements over standard therapies. The mean gain (measured by the Mann-Whitney statistic) was relatively constant across study designs, except for non-randomized controlled trials with sequential assignment to therapy, which showed a significantly higher likelihood that a patient would do better on the innovation than on standard therapy (p = 0.004). Randomized controlled trials that did not use a double-blind design had a higher likelihood of showing a gain for the innovation than did double-blind trials (p = 0.02). Any evaluation of an innovation may include both bias and the true efficacy of the new therapy, therefore we may consider making adjustments for the average bias associated with a study design. When interpreting an evaluation of a new therapy, readers should consider the impact of the following average adjustments to the Mann-Whitney statistic: for trials with non-random sequential assignment a decrease of 0.15, for non-double-blind randomized controlled trials a decrease of 0.11.

Racial differences in acute lymphocytic leukemia mortality and incidence trends

Mortality and morbidity rates for childhood leukemia are examined with reference to time trends and racial differences. Prior to 1964, white and nonwhite children had very different acute lymphocytic leukemia (ALL) mortality rates. With the advent of successful chemotherapy, the mortality rate of ALL in white children has decreased resulting in virtually no racial differences in ALL mortality. The reasons for a stable ALL mortality rate in nonwhite children despite successful chemotherapy include poor access to health care, undefined socioeconomic factors and a biologically different type of ALL occurring in nonwhite children. Recent data from two cancer surveys (1969-1971, 1973-1976) reveal that nonwhite children have a lower incidence of ALL than white children. Underreporting in the nonwhite children could partly account for the difference, but other contributing factors might include a genetic predisposition, undefined socioeconomic influences, and perhaps, a viral agent. Further studies of the factors contributing to racial differences in ALL incidence and mortality are needed.

The use of historical controls in breast cancer. An assessment in three consecutive trials

To evaluate the effectiveness of a new therapy some authors suggest the use of a retrospective control group, especially in cancer trials. In this article, by means of data collected in three consecutive randomized clinical trials on breast cancer performed at the Istituto Nazionale Tumori of Milan, the risks of a practice involving use of data from one trial as "historical controls" for the next one were investigated. The comparability between the baseline characteristics of different groups has been examined by resorting to univariate as well as multivariate statistical methods. The results of the final evaluation of the relative effectiveness of the treatments under study have also been investigated, using both randomized concurrent and historical controls. Although the eligibility criteria did not change, some differences between groups of consecutive trials were observed. The results attained by using historical control data, after adjusting for the differential composition in the baseline characteristics, made evident a biased conclusion in favor of the new treatment tested, which, in a historically controlled study, is always the last one. These results appear to be in agreement with those published by Sacks, Chalmers, and Smith (Am J Med 72:233-240, 1982).