Validity of Criteria Used to Evaluate Fingerstick Devices That Assess International Normalized Ratio

Novel analysis of clinically relevant diagnostic errors in point-of-care devices

BACKGROUND

To ensure proper clinical decision-making and avoid preventable harm, the quality of point-of-care (POC) device measures is routinely assessed. Traditional analyses may not reveal clinically important diagnostic errors.

OBJECTIVES

To compare results between a novel analytic framework and traditional analyses.

METHODS

Patients in four anticoagulation clinics provided two measures of the International Normalized Ratio (INR) at the same visit as part of routine quality assurance: one via a venous sample and one fingerstick. These were assessed with Hemochron POC devices. Traditional, quarterly, quality assurance assessments emphasized correlation analysis. The novel analysis used enhanced graphics and a validated assessment of clinical decision-making.

RESULTS

1518 paired INRs were analyzed. The correlation between the POC and laboratory assessments ranged between 0.84 and 0.91. Traditional quality assurance showed that the Hemochron devices were acceptable for continued use in each quarterly analysis. Enhanced graphical analysis demonstrated that the Hemochron devices never reported seven common INR values. The Hemochron devices systematically inflated values < 3 and deflated values > 4, biasing results towards the target INR range. Consequently, the Hemochron devices lead to a different clinical decision than the clinical laboratory measure in 31% of cases (458/1466; 95% confidence interval [CI] 29-34). When the reference INR was low, the Hemochron devices would not result in appropriate dose increases in 52% of cases (95% CI 48-56), placing these patients at risk for a significant adverse drug event.

CONCLUSIONS

Our novel, clinically relevant analysis revealed previously undetected deficiencies in our POC INR devices, and our approach should be adopted by industry, regulators, and institutions.

Clinical decision-making as the basis for assessing agreement between measures of the International Normalized Ratio

BACKGROUND

The now classic approach of Bland and Altman is often used to assess the level of agreement between International Normalized Ratio (INR) measures. However, we are concerned that this method does not define agreement in a clinically meaningful way. Agreement between measures should be characterized explicitly in terms of clinical decisions that result from INR measures.

OBJECTIVES

To develop and validate an extension of the Bland-Altman method to assess agreement between INR measures, based explicitly on the way clinicians make decisions.

METHODS AND RESULTS

We developed a clinically based graphical method to estimate the level of agreement between measures of INR. We identified clinically relevant INR ranges using epidemiologic and clinical evidence regarding risk and expected outcome at different INR ranges. Clinical decisions were expected to agree within these INR ranges and, therefore, the ranges became the basis for establishing agreement between measures. We used paired INR measures and resultant clinical decisions measured during a previous prospective study to validate and compare the accuracy of our model to those of Bland and Altman's and other published models. Our method more accurately predicts when warfarin dosing decisions differ than the Bland-Altman method (P < 0.02). Our method is also superior to other published methods, particularly at the important task of identifying when measures lead to discrepant clinical decisions.

CONCLUSIONS

We introduced and validated an improvement of the Bland-Altman method to assess agreement between INR measures. Our model is superior because it is based explicitly on factors that influence clinical decision-making.