Statistical inference for the within-device precision of quantitative measurements in assay validation

Measuring precision in bioassays: Rethinking assay validation

The m:n:θ_b procedure is often used for validating an assay for precision, where m levels of an analyte are measured with n replicates at each level, and if all m estimates of coefficient of variation (CV) are less than θ_b , then the assay is declared validated for precision. The statistical properties of the procedure are unknown so there is no clear statistical statement of precision upon passing. Further, it is unclear how to modify the procedure for relative potency assays in which the constant standard deviation (SD) model fits much better than the traditional constant CV model. We use simple normal error models to show that under constant CV across the m levels, the probability of passing when the CV is θ_b is about 10% to 20% for some recommended implementations; however, for extreme heterogeniety of CV when the largest CV is θ_b , the passing probability can be greater than 50%. We derive 100q% upper confidence limits on the CV under constant CV models and derive analogous limits for the SD under a constant SD model. Additionally, for a post-validation assay output of y, we derive 68.27% confidence intervals on either the mean or log geometric mean of the assay output using either y±s (for the constant SD model) or log(y)±rG (for the constant CV model), where s and r_G are constants that do not depend on y. We demonstrate the methods on a growth inhibition assay used to measure biologic activity of antibodies against the malaria parasite.

Testing equality of generalized treatment effects

Recently, the concept of generalized treatment effect, defined as P(X > Y) where X and Y denote continuous outcome variables for treatment arm and control arm, respectively, has been proposed as an appropriate measure of treatment effect in clinical trials with parallel design. Compared to the mean difference, the generalized treatment effect has many advantages; for example, it is a scaleless measure and it does not change under monotonic transformations. This article investigates the problem of testing equality of generalized treatment effects among several clinical trials. The proposed approach follows the same vein as the generalized variable method for testing equality of several log-normal means proposed by Li (2009). Numerical study demonstrates that the proposed test has excellent type I error control for clinical trials with small to medium sample sizes. Robustness study shows that the proposed method performs reasonably for categorical data.