Sample Sizes Needed for Specified Margins of Relative Error in the Estimates of the Repeatability and Reproducibility Standard Deviations

Sample size formulas are developed to estimate the repeatability and reproducibility standard deviations (sr and sR) such that the actual error in (sr and sR) relative to their respective true values, σr and σR, are at predefined levels. The statistical consequences associated with AOAC INTERNATIONAL required sample size to validate an analytical method are discussed. In addition, formulas to estimate the uncertainties of (sr and sR) were derived and are provided as supporting documentation.Formula for the Number of Replicates Required for a Specified Margin of Relative Error in the Estimate of the Repeatability Standard Deviation

Validity of the Percent Reduction in Standard Deviation Outlier Test for Screening Laboratory Means from a Collaborative Study

AOAC INTERNATIONAL recommends an outlier test that is based on the “percent reduction in standard deviation” (PRSD) for screening the highest or lowest and 2-highest or 2-lowest laboratory means from a collaborative study. The original critical values that were developed to assess the significance of the test statistics associated with the PRSD test were obtained by simulation. In this paper, we answer several questions relative to the validity of the simulated critical values and develop formulas, based on the Student's t-distribution, to compare the simulated and formula-based critical values. Assumptions and derivations of formulas are provided along with selected tabular critical values that are used to test hypotheses at various levels of significance.

Computation of HORRAT Values

The formula for the Horwitz ratio (HORRAT) as presented in the Study Director's Manual of AOAC INTERNATIONAL is applicable only when the concentration is in the unit/unit form (e.g., μg/μg, g/g, etc.). When the analyte concentration is a trace or mass fraction amount (e.g., μg/g), the formula generates incorrect HORRAT values. Alternative calculation procedures are presented to circumvent such problems.

Use of prediction methods to assess laboratory bias and mean values associated with an interlaboratory study for method validation and/or proficiency testing

Two methods of prediction of random variables, best predictor (BP) and best linear unbiased predictor (BLUP), are discussed as potential statistical methods to predict laboratory true mean and bias values using the sample laboratory mean (y(i)) from interlaboratory studies. The predictions developed here require that the interlaboratory and/or proficiency study be designed and conducted in a manner consistent with the assumptions of a one-way completely randomized model (CRM). Under the CRM the individual laboratory true mean and bias are not parameters but are defined to be random variables that are unobservable and considered as realized values that cannot be estimated but can be predicted using methods of "prediction." The BP method is applicable when all salient parameters are known, e.g., the consensus true overall mean (mu) and repeatability and reproducibility components (sigma2(r) and sigma2(R)), while the BLUP method is useful when sigma2(r) and sigma2(R) are known, but mu is estimated by the generalized least square estimator. Although the derivations of predictors are obtained by minimizing the mean-square error under the CRM assumptions, the predictors are the expected laboratory true mean and bias given the sample laboratory mean, i.e., conditional expectation.

Determination of Operating Characteristic, Retesting, and Testing Amount Probabilities Associated with Testing for the Presence of Salmonella in Foods

The relatively small perceived probability associated with retesting a food for the presence of Salmonella at low levels is often considered as one of the reasons that a confirmatory or check-analysis tends to disagree in practice with the results of an original test. Given a retesting process where a retest is only performed to confirm an original positive Salmonella test, the probability that both the original and retest will test positive for Salmonella has been traditionally determined by some as the product of the probabilities of a positive Salmonella test for the original and retest samples. When examining the probabilities associated with the retesting process, we found that our results disagreed with those based on intuitions apparently held by others concerning how these probabilities should be calculated. For Salmonella testing, operating characteristic values were computed to demonstrate the protections afforded by the Salmonella sampling plans presented in the U.S. Food and Drug Administration's Bacteriological Analytical Manual and to obtain the probability of a positive Salmonella test. The geometric distribution was examined for possible utility in determining the probabilities associated with testing amounts, i.e., the number of Salmonella tests needed to obtain a positive test.

Determination of operating characteristic, retesting, and testing amount probabilities associated with testing for the presence of Salmonella in foods

The relatively small perceived probability associated with retesting a food for the presence of Salmonella at low levels is often considered as one of the reasons that a confirmatory or check-analysis tends to disagree in practice with the results of an original test. Given a retesting process where a retest is only performed to confirm an original positive Salmonella test, the probability that both the original and retest will test positive for Salmonella has been traditionally determined by some as the product of the probabilities of a positive Salmonella test for the original and retest samples. When examining the probabilities associated with the retesting process, we found that our results disagreed with those based on intuitions apparently held by others concerning how these probabilities should be calculated. For Salmonella testing, operating characteristic values were computed to demonstrate the protections afforded by the Salmonella sampling plans presented in the U.S. Food and Drug Administration's Bacteriological Analytical Manual and to obtain the probability of a positive Salmonella test. The geometric distribution was examined for possible utility in determining the probabilities associated with testing amounts, i.e., the number of Salmonella tests needed to obtain a positive test.

Determination of a two-tailed 100(1-alpha)% upper limit on the relative error in the laboratory-to-laboratory standard deviation obtained from an interlaboratory study

For some classes of analytical methods, it is assumed that the error in the laboratory-to-laboratory standard deviation (S(L)) is appreciable. To demonstrate the magnitude of this error in S(L) for such methods, formulas were derived to obtain a two-tailed 100(1-alpha)% upper limit on the relative error in S(L) obtained from an interlaboratory study, assuming that the laboratory-to-laboratory variance (S(L)2) obtained in the validation of an analytical method is approximately normal and/or Chi-square distributed. This 100(1-alpha)% upper limit (delta(1-alpha/2)) is referred to as a margin of relative error in S(L) (MRE(S(L. Monte Carlo simulations were performed, and the results compared satisfactorily with the formula calculations. To aid in designing future interlaboratory studies in which concern is focused on the magnitude of the uncertainty in S(L), expressed as a proportion of the true value (sigma L), a formula was derived to determine the number of laboratories needed to attain a given MRE in S(L) for a stated number of replicates per laboratory.

Determination of a Two-Tailed 100(1-α)% Upper Limit on the Relative Error in the Laboratory-to-Laboratory Standard Deviation Obtained from an Interlaboratory Study

For some classes of analytical methods, it is assumed that the error in the laboratory-to-laboratory standard deviation (sL) is appreciable. To demonstrate the magnitude of this error in sL for such methods, formulas were derived to obtain a two-tailed 100(1-α)% upper limit on the relative error in sL obtained from an interlaboratory study, assuming that the laboratory-to-laboratory variance (sL2) obtained in the validation of an analytical method is approximately normal and/or Chi-square distributed. This 100(1-α)% upper limit is referred to as a margin of relative error in sL (MRE(sL)). Monte Carlo simulations were performed, and the results compared satisfactorily with the formula calculations. To aid in designing future interlaboratory studies in which concern is focused on the magnitude of the uncertainty in sL, expressed as a proportion of the true value (σL), a formula was derived to determine the number of laboratories needed to attain a given MRE in sL for a stated number of replicates per laboratory.

On using a normal approximation for the noncentral t-distribution in determining upper limits for future sample relative repeatability and reproducibility standard deviations

Formulas, based on a normal approximation for the noncentral t-distribution, were developed to compute 100p% one-tailed upper limits for future sample relative repeatability and relative reproducibility standard deviations (RSDr,% and RSDR,%) collaboratively obtained under a completely randomized model. The accuracy of the formulas for obtaining a one-tailed upper limit for the future sample RSDr,% was assessed by comparing the computed noncentral t-distribution-based upper limits with the one-tailed upper limits based on a normal approximation for the noncentral t-distribution. The accuracy of the normal approximation formula for obtaining a one-tailed upper limit for a future sample RSDR,% was assessed by comparing the formula-based one-tailed upper limits with those obtained in a Monte Carlo simulation study.

On Using a Normal Approximation for the Noncentral t-Distribution in Determining Upper Limits for Future Sample Relative Repeatability and Reproducibility Standard Deviations

Formulas, based on a normal approximation for the noncentral t-distribution, were developed to compute 100p one-tailed upper limits for future sample relative repeatability and relative reproducibility standard deviations (RSDr, and RSDR,) collaboratively obtained under a completely randomized model. The accuracy of the formulas for obtaining a one-tailed upper limit for the future sample RSDr, was assessed by comparing the computed noncentral t-distribution-based upper limits with the one-tailed upper limits based on a normal approximation for the noncentral t-distribution. The accuracy of the normal approximation formula for obtaining a one-tailed upper limit for a future sample RSDR, was assessed by comparing the formula-based one-tailed upper limits with those obtained in a Monte Carlo simulation study.

Uncertainties of method performance statistics based on a balanced completely randomized model interlaboratory study

Using exact and asymptotic distributions, formulas were developed to derive population variances and uncertainties, and their corresponding unbiased estimates, for the method performance statistics, i.e., the sample mean (y..), the repeatability variance and standard deviation and (s2r and sr), and the reproducibility variance and standard deviation (s2R and sR) and obtained from collaborative study of an analytical method.

Uncertainties of Method Performance Statistics Based on a Balanced Completely Randomized Model Interlaboratory Study

Using exact and asymptotic distributions, formulas were developed to derive population variances and uncertainties, and their corresponding unbiased estimates, for the method performance statistics, i.e., the sample mean (y_bar..), the repeatability variance and standard deviation and (sr2 and sr), and the reproducibility variance and standard deviation (sR2 and sR) and obtained from collaborative study of an analytical method.

Exact One-Tailed 100p Upper Limits for Future Sample Repeatability Relative Standard Deviations Obtained in Single and Multilaboratory Repeatability Studies

Formulas are derived to obtain exact one-tailed 100p upper limits (p and p) for future sample repeatability relative standard deviations, based on a noncentral t-distribution, for multi- and single-laboratory repeatability studies, respectively, used in the validation of analytical methods.

Exact one-tailed 100p% upper limits for future sample repeatability relative standard deviations obtained in single- and multilaboratory repeatability studies

Formulas are derived to obtain exact one-tailed 100p% upper limits (kappa(p) and nu(p)) for future sample repeatability relative standard deviations, based on a noncentral t-distribution, for multi- and single-laboratory repeatability studies, respectively, used in the validation of analytical methods.

On Using an Approximate Noncentral t-Distribution in Determining a One-Side Upper Limit for Future Sample Relative Reproducibility Standard Deviations

For future sample relative reproducibility standard deviations (RSDR), collaboratively obtained under a completely randomized model (CRM), a new formula for determining a one-tailed 100p% upper limit (p) for such RSDR values was developed based on an approximate noncentral t-distribution with degrees of freedom obtained using Satterthwaite's adjustment. The accuracy of p was assessed by comparing p and its probability levels with similar values associated with a Monte Carlo simulation and with those obtained using another formula (p) that was developed for the same purpose but based on a normal approximation.

On using an approximate noncentral t-distribution in determining a one-side upper limit for future sample relative reproducibility standard deviations

For future sample relative reproducibility standard deviations (RSD(R)), collaboratively obtained under a completely randomized model (CRM), a new formula for determining a one-tailed 100p% upper limit (kappap) for such RSD(R) values was developed based on an approximate noncentral t-distribution with degrees of freedom obtained using Satterthwaite's adjustment. The accuracy of kappap was assessed by comparing kappap and its probability levels with similar values associated with a Monte Carlo simulation and with those obtained using another formula (gammap) that was developed for the same purpose but based on a normal approximation.

On Determining a 1-Tailed Upper Limit for Future Sample HorRat Values

Two formulas were developed for use in computing 1-tailed upper limits for future HorRat values obtained from the collaborative study of materials. One formula is applicable when a future sample HorRat value <inline-formula> <inline-graphic href="inline_eq1.gif"/> </inline-formula> is computed based on a known concentration (e.g., C spike level and RSDR is the sample relative reproducibility standard deviation) and the other formula is applicable when the true concentration (C) is unknown and a future sample HorRat value <inline-formula> <inline-graphic href="inline_eq2.gif"/> </inline-formula> is computed using the sample mean (e.g., <inline-formula> <inline-graphic href="inline_eq3.gif"/> </inline-formula>, the collaborative study overall mean for an analyte). A Monte Carlo simulation procedure was developed using the Statistical Analysis System (SAS) software to assess the accuracy of the 2 developed formulas. Based on the degree of closeness between the simulated and calculated limits, the formulas for computing upper limits for future sample HorRat values will prove to be useful to Study Directors in determining worst case scenarios concerning a method's reproducibility precision relative to that predicted using the Horwitz equation. We also define the current empirical HorRat limits as 1-tailed 100p% upper limits to assess the statistical consequence, in a probability sense, of their application as an analytical methods screening tool.

On determining a 1-tailed upper limit for future sample HorRat values

Two formulas were developed for use in computing 1-tailed upper limits for future HorRat values obtained from the collaborative study of materials. One formula is applicable when a future sample HorRat value H [formula: see text] is computed based on a known concentration (e.g., C = spike level and RSD(R) is the sample relative reproducibility standard deviation) and the other formula is applicable when the true concentration (C) is unknown and a future sample HorRat value [formula: see text] is computed using the sample mean (e.g., y, the collaborative study overall mean for an analyte). A Monte Carlo simulation procedure was developed using the Statistical Analysis System (SAS) software to assess the accuracy of the 2 developed formulas. Based on the degree of closeness between the simulated and calculated limits, the formulas for computing upper limits for future sample HorRat values will prove to be useful to Study Directors in determining worst case scenarios concerning a method's reproducibility precision relative to that predicted using the "Horwitz equation". We also define the current empirical HorRat limits as 1-tailed 100p% upper limits to assess the statistical consequence, in a probability sense, of their application as an analytical methods screening tool.

Determining a One-Tailed Upper Limit for Future Sample Relative Reproducibility Standard Deviations

A formula was developed to determine a one-tailed 100p% upper limit for future sample percent relative reproducibility standard deviations <inline-formula> <inline-graphic href="inline_eq1.gif"/> </inline-formula> , where sR is the sample reproducibility standard deviation, which is the square root of a linear combination of the sample repeatability variance ( <inline-formula> <inline-graphic href="inline_eq2.gif"/> </inline-formula> ) plus the sample laboratory-to-laboratory variance ( <inline-formula> <inline-graphic href="inline_eq3.gif"/> </inline-formula> ), i.e., <inline-formula> <inline-graphic href="inline_eq4.gif"/> </inline-formula> , and y is the sample mean. The future RSDR, % is expected to arise from a population of potential RSDR, % values whose true mean is <inline-formula> <inline-graphic href="inline_eq5.gif"/> </inline-formula> , where R and are the population reproducibility standard deviation and mean, respectively.

Determining a one-tailed upper limit for future sample relative reproducibility standard deviations

A formula was developed to determine a one-tailed 100p% upper limit for future sample percent relative reproducibility standard deviations (RSD(R),%= 100s(R)/y), where S(R) is the sample reproducibility standard deviation, which is the square root of a linear combination of the sample repeatability variance (s(r)2) plus the sample laboratory-to-laboratory variance (s(L)2), i.e., S(R) = s(L)2, and y is the sample mean. The future RSD(R),% is expected to arise from a population of potential RSD(R),% values whose true mean is zeta(R),% = 100sigmaR, where sigmaR and mu are the population reproducibility standard deviation and mean, respectively.

Sample sizes needed for specified margins of relative error in the estimates of the repeatability and reproducibility standard deviations

Sample size formulas are developed to estimate the repeatability and reproducibility standard deviations (Sr and S(R)) such that the actual error in (Sr and S(R)) relative to their respective true values, sigmar and sigmaR, are at predefined levels. The statistical consequences associated with AOAC INTERNATIONAL required sample size to validate an analytical method are discussed. In addition, formulas to estimate the uncertainties of (Sr and S(R)) were derived and are provided as supporting documentation. Formula for the Number of Replicates Required for a Specified Margin of Relative Error in the Estimate of the Repeatability Standard Deviation.

Validity of the percent reduction in standard deviation outlier test for screening laboratory means from a collaborative study

AOAC INTERNATIONAL recommends an outlier test that is based on the "percent reduction in standard deviation" (PRSD) for screening the highest or lowest and 2-highest or 2-lowest laboratory means from a collaborative study. The original critical values that were developed to assess the significance of the test statistics associated with the PRSD test were obtained by simulation. In this paper, we answer several questions relative to the validity of the simulated critical values and develop formulas, based on the Student's t-distribution, to compare the simulated and formula-based critical values. Assumptions and derivations of formulas are provided along with selected tabular critical values that are used to test hypotheses at various levels of significance.

Determination of sample size for validation of allergen-screening methods

For various levels of confidence (i.e., 80 and 90%) and ratios (K = sigmap2/sigmaN2, where sigmap2 and sigmaN2 are the analyte variances for the positive and negative distributions, respectively), sample sizes sufficient to test the requirements that a given method detects > or = 90% of the positives (> or = 5 ppm of a given analyte) while misclassifying < or = 10% of the negatives (implying a specificity rate, true negatives that will be correctly classified, of 90%) were estimated by using a rationale that minimizes the cost of sampling.

Computation of HORRAT values

The formula for the Horwitz ratio (HORRAT) as presented in the Study Director's Manual of AOAC INTERNATIONAL is applicable only when the concentration is in the unit/unit form (e.g., microg/microg, g/g, etc.). When the analyte concentration is a trace or mass fraction amount (e.g., microg/g), the formula generates incorrect HORRAT values. Alternative calculation procedures are presented to circumvent such problems.

Determination of Sample Size for Validation of Allergen-Screening Methods

For various levels of confidence (i.e., 80 and 90%) and ratios (K = σP2/σN2 where σP2 and σN2 are the analyte variances for the positive and negative distributions, respectively), sample sizes sufficient to test the requirements that a given method detects ≥90% of the positives (≥5 ppm of a given analyte) while misclassifying ≤10% of the negatives (implying a specificity rate, true negatives that will be correctly classified, of 90%) were estimated by using a rationale that minimizes the cost of sampling.

Procedures for estimating confidence intervals for selected method performance parameters

Procedures for estimating confidence intervals (CIs) for the repeatability variance (sigmar2), reproducibility variance (sigmaR2 = sigmaL2 + sigmar2), laboratory component (sigmaL2), and their corresponding standard deviations sigmar, sigmaR, and sigmaL, respectively, are presented. In addition, CIs for the ratio of the repeatability component to the reproducibility variance (sigmar2/sigmaR2) and the ratio of the laboratory component to the reproducibility variance (sigmaL2/sigmaR2) are also presented.

A statistical model to evaluate analyte homogeneity for a material

An underlying assumption for collaborative studies is that the analyte variation among test samples of the material (i.e., matrix and analyte concentration combination) under study has a negligible influence on the estimates of precision for the method. This assumption is expected to be fulfilled when the material under study is prepared (i.e., thoroughly mixed) such that the analyte is distributed uniformly throughout the matrix. Statistical design and intra-class correlation analysis procedures are proposed to assess the similarity or agreement among analytical results among- and within-containers for single and multiple occasions of use (e.g., collaborative and proficiency studies).

A Statistical Model To Evaluate Analyte Homogeneity for a Material

An underlying assumption for collaborative studies is that the analyte variation among test samples of the material (i.e., matrix and analyte concentration combination) under study has a negligible influence on the estimates of precision for the method. This assumption is expected to be fulfilled when the material under study is prepared (i.e., thoroughly mixed) such that the analyte is distributed uniformly throughout the matrix. Statistical design and intra-class correlation analysis procedures are proposed to assess the similarity oragreement among analytical results among- and within-containers for single and multiple occasions of use (e.g., collaborative and proficiency studies).

Procedures for Estimating Confidence Intervals for Selected Method Performance Parameters

Procedures for estimating confidence intervals (CIs) for the repeatability variance (σr2), reproducibility variance (σR2 = σL2 + σr2), laboratory component (σL2), and their corresponding standard deviations σr, σR, and σL, respectively, are presented. In addition, CIs for the ratio of the repeatability component to the reproducibility variance (σr2/σR2) and the ratio of the laboratory component to the reproducibility variance (σL2/σR2) are also presented.

A statistical evaluation of the Youden Matched-Pairs Procedure

AOAC INTERNATIONAL currently permits investigators to use the Youden Matched-Pairs Procedure to obtain estimates of the method performance indicators repeatability and reproducibility (S(r) and SR, respectively). This report explains the statistical model assumptions upon which the procedure is based, provides validity tests for several of these assumptions, explains conditions under which Youdens' "precision error" is not consistent with precision estimate S(r) or SR as defined by AOAC INTERNATIONAL, and indicates when precision estimates based on the procedure should be interpreted with caution or should not be used.

A Statistical Evaluation of the Youden Matched-Pairs Procedure

AOAC INTERNATIONAL currently permits investigators to use the Youden Matched-Pairs Procedure to obtain estimates of the method performance indicators repeatability and reproducibility (sr and sR, respectively). This report explains the statistical model assumptions upon which the procedure is based, provides validity tests for several of these assumptions, explains conditions under which Youdens’ “precision error” is not consistent with precision estimate sr or sR as defined by AOAC INTERNATIONAL, and indicates when precision estimates based on the procedure should be interpreted with caution or should not be used.

Repeatability and reproducibility estimates from collaborative studies based on total concentration of trace analytes

In the process of validating a given analytical method for the total concentration of a trace analyte, the precision indicators, repeatability and reproducibility, are obtained from a collaborative study of the method based on a standard one-way completely randomized model. This report discusses the shortcomings of the statistical models used in such studies, defines the component makeup for estimates of the repeatability and reproducibility variances based on these models, and considers suggestions offered as new policy regarding method performance based on total concentration.

A precollaborative study of weight determination methods for quick frozen shrimp

A precollaborative study compared the accuracy and precision of official AOAC methods with other selected methods for determining net weight of IQF-glazed shrimp and block-glazed shrimp, assessed the ruggedness of the methods with respect to changes in the levels of the factors under study, and selected candidate methods for use in a collaborative study. Methods tested for determining deglazed (frozen) net weight of IQF-glazed shrimp were (1) AOAC Method 963.18 and (2) the Water Bath Dip Method. Methods tested for determining thawed net weight of IQF-glazed shrimp were (1) AOAC Method 967.13, (2) Modified AOAC Method(mnb) 967.13, (3) Modified AOAC Method(pb) 967.13, (4) the Codex Method, (5) the Air Thaw Method, and (6) Modified AOAC Method 963.18. The same methods except Modified AOAC Method 963.18 were tested for determining thawed net weight of block-glazed shrimp. A total of 864 0.45 kg (1 lb), 0.90 kg (2 lb), and 1.35 kg (3 lb) IQF-glazed shrimp test samples and 234 2.25 kg (5 lb) block-glazed shrimp test samples were collected. During sample preparation, test samples were subjected to either water with or without sodium tripolyphosphate (STP). During deglazing (IQF-glazed shrimp only) and/or thawing, test samples were allocated in a factorial design to assess the effects of STP presence (no STP and STP), sieve mesh sizes (2.83 and 2.38 mm; 0.11 and 0.09 in.), and sieve diameters (20 and 30 cm; 8 and 12 in.). During weighing, test samples were further allocated to a sequence of weighing procedures designed to assess the effects of using sieve weights (dry and wet) in combination with paper towel use (no and yes) and tared pan weights when calculating determined net weights. On the basis of the results of this precollaborative study, Modified AOAC Method(pb) 967.13 and the Air Thaw Method seem to be the best methods to determine net weight of IQF-glazed and block-glazed shrimp. Therefore, to validate method choices in the collaborative study, the authors recommend analysis of IQF-glazed shrimp and block-glazed shrimp test samples, each prepared with or without STP, by Modified AOAC Method(pb) 967.13 and the Air Thaw Method. To fulfill AOAC requirements, IQF-glazed shrimp and block-glazed shrimp test samples, each prepared with or without STP, must be analyzed by official methods: AOAC Method 963.18 (IQF-glazed shrimp only) and AOAC Method 967.13. During testing, sieve mesh size will be either 2.83 or 2.38 mm (0.11 or 0.09 in.), sieve diameter will be limited to 30 cm (12 in.), and weighing procedure will be limited to tared pan.

Determination of fish flesh content in frozen coated fish products (modification of AOAC Official Method 971.13): Collaborative Study

An intralaboratory collaborative study evaluated a modified version of AOAC Official Method 971.13 for determining the fish flesh content (FFC) in frozen coated fish products by comparing it with the on-line method. Eleven collaborators analyzed 36 products (a total of 6336 test samples). Each product targeted one of 4 percent fish flesh (PFF) levels (35, 50, 65, and 80). Products were manufactured from one of 3 raw materials (fillet blocks, minced blocks, and natural fillets) and processed in one of 4 forms (sticks, portions, formed portions, and fillets) and one of 4 styles (raw breaded, batter-dipped, precooked, and fully cooked). Each "official" test sample was tracked through the processing system and weighed (1) before battering and/or breading and, depending on product style, before frying; and (2) after battering and/or breading and, depending on product style, after frying; so that it served as its own control. These weights were used to calculate actual percent fish flesh (APFF) and considered to be generated by the on-line method. Collaborators weighed official test samples (1) before scraping; and (2) after scraping. These weights were used to calculate determined percent fish flesh (DPFF) and considered to be generated by the modified AOAC method. APFF and DPFF were the primary data for statistical analysis. Recoveries ranged from 71.75 to 106.40%. Repeatability (method precision indicator for a single collaborators) relative standard deviation (RSDr) values ranged from 1.04 to 8.37%. Corresponding reproducibility (method precision indicator among collaborators) relative standard deviation (RSDR) values ranged from 1.41 to 11.95%. The DPFF mean was lower than the APFF mean for 30 (83.3%) of 36 products. For 29 of these 30 products, the differences between method means (APFF minus DPFF) ranged from 0.38 to 6.51%. For the remaining product within this group (C/06, fillet blocks, fully cooked portions), the difference between method means was 21.73%. For the remaining 6 (16.67%) of 36 products, the DPFF mean was greater than the APFF mean. The differences between method means ranged from -0.03 to -2.76%. RSD values were considered acceptable (i.e., RDSr < 9% and RSDR < 12%) for all products studied. The modified method for determining FFC in frozen coated fish products has been adopted first action to replace AOAC Official Method 971.13.