Effects of and preference for pay for performance: an analogue analysis

Randomization Tests for Single Case Designs with Rapidly Alternating Conditions: An Analysis of p-Values from Published Experiments

Two common barriers to applying statistical tests to single-case experiments are that single-case data often violate the assumptions of parametric tests and that random assignment is inconsistent with the logic of single-case design. However, in the case of randomization tests applied to single-case experiments with rapidly alternating conditions, neither the statistical assumptions nor the logic of the designs are violated. To examine the utility of randomization tests for single-case data, we collected a sample of published articles including alternating treatments or multielement designs with random or semi-random condition sequences. We extracted data from graphs and used randomization tests to estimate the probability of obtaining results at least as extreme as the results in the experiment by chance alone (i.e., p-value). We compared the distribution of p-values from experimental comparisons that did and did not indicate a functional relation based on visual analysis and evaluated agreement between visual and statistical analysis at several levels of α. Results showed different means, shapes, and spreads for the p-value distributions and substantial agreement between visual and statistical analysis when α = .05, with lower agreement when α was adjusted to preserve family-wise error at .05. Questions remain, however, on the appropriate application and interpretation of randomization tests for single-case designs.

Reinforcing Productivity in a Job-Skills Training Program for Unemployed Substance-Abusing Adults

Chronically unemployed adults may benefit from intensive job-skills training; however, training programs do not always reliably engage participants in mastering skills. This study examined effects of voucher reinforcement for performance on a job-skills training program in the therapeutic workplace. Participants were four unemployed, substance abusing adults who earned monetary vouchers for working on programs targeting typing skills. Participants were exposed to two payment conditions that differed in whether or not pay was dependent on performance in a within-subject reversal design. In the productivity-pay condition, participants earned $8.00 per hour for attending the workplace plus a bonus for performance. In the base-pay condition, participants were paid an hourly wage that was equivalent to the total hourly earnings from the previous productivity-pay condition. Participants completed less work on the typing programs in the base- than the productivity-pay condition, but the amount of time spent in the workroom and the accuracy and rate of typing were not affected by the pay manipulation. All participants reported preferring base pay over productivity pay. Explicit reinforcement of productivity maintains consistent work in training programs, but more aspects of productivity pay need to be refined for effective, efficient, and socially valid implementation with unemployed, substance-abusing adults.

Effectiveness of iterative interventions to increase research productivity in one residency program

BACKGROUND

The Accreditation Council for Graduate Medical Education requires residency programs to expose residents to research opportunities.

OBJECTIVE

The purpose of this study was to assess the impact of a series of iterative interventions to increase scholarly activity in one internal medicine residency.

METHODS

Retrospective analysis of the effectiveness of a series of interventions to increase resident and faculty scholarly productivity over a 14-year period was performed using quality improvement methodology. Outcomes measured were accepted regional and national abstracts and PubMed indexed manuscripts of residents and faculty.

RESULTS

Initially, regional meeting abstracts increased and then were supplanted by national meeting abstracts. Sustained gains in manuscript productivity occurred in the eighth year of interventions, increasing from a baseline of 0.01 publications/FTE/year to 1.57 publications/FTE/year in the final year measured. Run chart analysis indicated special cause variation associated with the interventions performed.

CONCLUSIONS

Programs attempting to stimulate research production among faculty and residents can choose among many interventions cited in the literature. Since success of any group of interventions is likely additive and may take years to show benefit, measuring outcomes using quality improvement methodology may be an effective way to determine success.