The effect of clustering on lot quality assurance sampling: a probabilistic model to calculate sample sizes for quality assessments

Mixed Methods Lot Quality Assurance Sampling: A novel, rapid methodology to inform equity focused maternal health programming in rural Rajasthan, India

India has experienced a significant increase in facility-based delivery (FBD) coverage and reduction in maternal mortality. Nevertheless, India continues to have high levels of maternal health inequity. Improving equity requires data collection methods that can produce a better contextual understanding of how vulnerable populations access and interact with the health care system at a local level. While large population-level surveys are valuable, they are resource intensive and often lack the contextual specificity and timeliness to be useful for local health programming. Qualitative methods can be resource intensive and may lack generalizability. We describe an innovative mixed-methods application of Large Country-Lot Quality Assurance Sampling (LC-LQAS) that provides local coverage data and qualitative insights for both FBD and antenatal care (ANC) in a low-cost and timely manner that is useful for health care providers working in specific contexts. LC-LQAS is a version of LQAS that combines LQAS for local level classification with multistage cluster sampling to obtain precise regional or national coverage estimates. We integrated qualitative questions to uncover mothers' experiences accessing maternal health care in the rural district of Sri Ganganagar, Rajasthan, India. We interviewed 313 recently delivered, low-income women in 18 subdistricts. All respondents participated in both qualitative and quantitative components. All subdistricts were classified as having high FBD coverage with the upper threshold set at 85%, suggesting that improved coverage has extended to vulnerable women. However, only two subdistricts were classified as high ANC coverage with the upper threshold set at 40%. Qualitative data revealed a severe lack of agency among respondents and that household norms of care seeking influenced uptake of ANC and FBD. We additionally report on implementation outcomes (acceptability, feasibility, appropriateness, effectiveness, fidelity, and cost) and how study results informed the programs of a local health non-profit.

Use of Lot quality assurance sampling surveys to evaluate community health worker performance in rural Zambia: a case of Luangwa district

BACKGROUND

The Better Health Outcomes through Mentoring and Assessment (BHOMA) project is a cluster randomized controlled trial aimed at reducing age-standardized mortality rates in three rural districts through involvement of Community Health Workers (CHWs), Traditional Birth Attendants (TBAs), and Neighborhood Health Committees (NHCs). CHWs conduct quarterly surveys on all households using a questionnaire that captures key health events occurring within their catchment population. In order to validate contact with households, we utilize the Lot Quality Assurance Sampling (LQAS) methodology. In this study, we report experiences of applying the LQAS approach to monitor performance of CHWs in Luangwa District.

METHODS

Between April 2011 and December 2013, seven health facilities in Luangwa district were enrolled into the BHOMA project. The health facility catchment areas were divided into 33 geographic zones. Quality assurance was performed each quarter by randomly selecting zones representing about 90% of enrolled catchment areas from which 19 households per zone where also randomly identified. The surveys were conducted by CHW supervisors who had been trained on using the LQAS questionnaire. Information collected included household identity number (ID), whether the CHW visited the household, duration of the most recent visit, and what health information was discussed during the CHW visit. The threshold for success was set at 75% household outreach by CHWs in each zone.

RESULTS

There are 4,616 total households in the 33 zones. This yielded a target of 32,212 household visits by community health workers during the 7 survey rounds. Based on the set cutoff point for passing the surveys (at least 75% households confirmed as visited), only one team of CHWs at Luangwa high school failed to reach the target during round 1 of the surveys; all the teams otherwise registered successful visits in all the surveys.

CONCLUSIONS

We have employed the LQAS methodology for assurance that quarterly surveys were successfully done. This methodology proved helpful in identifying poorly performing CHWs and could be useful for evaluating CHW performance in other areas.

TRIAL REGISTRATION

Identifier: NCT01942278 . Date of Registration: September 2013.

Data for Program Management: An Accuracy Assessment of Data Collected in Household Registers by Community Health Workers in Southern Kayonza, Rwanda

Community health workers (CHWs) collect data for routine services, surveys and research in their communities. However, quality of these data is largely unknown. Utilizing poor quality data can result in inefficient resource use, misinformation about system gaps, and poor program management and effectiveness. This study aims to measure CHW data accuracy, defined as agreement between household registers compared to household member interview and client records in one district in Eastern province, Rwanda. We used cluster-lot quality assurance sampling to randomly sample six CHWs per cell and six households per CHW. We classified cells as having 'poor' or 'good' accuracy for household registers for five indicators, calculating point estimates of percent of households with accurate data by health center. We evaluated 204 CHW registers and 1,224 households for accuracy across 34 cells in southern Kayonza. Point estimates across health centers ranged from 79 to 100% for individual indicators and 61 to 72% for the composite indicator. Recording error appeared random for all but the widely under-reported number of women on modern family planning method. Overall, accuracy was largely 'good' across cells, with varying results by indicator. Program managers should identify optimum thresholds for 'good' data quality and interventions to reach them according to data use. Decreasing variability and improving quality will facilitate potential of these routinely-collected data to be more meaningful for community health program management. We encourage further studies assessing CHW data quality and the impact training, supervision and other strategies have on improving it.

Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys

Lot quality assurance sampling (LQAS) surveys are commonly used for monitoring and evaluation in resource-limited settings. Recently several methods have been proposed to combine LQAS with cluster sampling for more timely and cost-effective data collection. For some of these methods, the standard binomial model can be used for constructing decision rules as the clustering can be ignored. For other designs, considered here, clustering is accommodated in the design phase. In this paper, we compare these latter cluster LQAS methodologies and provide recommendations for choosing a cluster LQAS design. We compare technical differences in the three methods and determine situations in which the choice of method results in a substantively different design. We consider two different aspects of the methods: the distributional assumptions and the clustering parameterization. Further, we provide software tools for implementing each method and clarify misconceptions about these designs in the literature. We illustrate the differences in these methods using vaccination and nutrition cluster LQAS surveys as example designs. The cluster methods are not sensitive to the distributional assumptions but can result in substantially different designs (sample sizes) depending on the clustering parameterization. However, none of the clustering parameterizations used in the existing methods appears to be consistent with the observed data, and, consequently, choice between the cluster LQAS methods is not straightforward. Further research should attempt to characterize clustering patterns in specific applications and provide suggestions for best-practice cluster LQAS designs on a setting-specific basis.

Extending cluster lot quality assurance sampling designs for surveillance programs

Lot quality assurance sampling (LQAS) has a long history of applications in industrial quality control. LQAS is frequently used for rapid surveillance in global health settings, with areas classified as poor or acceptable performance on the basis of the binary classification of an indicator. Historically, LQAS surveys have relied on simple random samples from the population; however, implementing two-stage cluster designs for surveillance sampling is often more cost-effective than simple random sampling. By applying survey sampling results to the binary classification procedure, we develop a simple and flexible nonparametric procedure to incorporate clustering effects into the LQAS sample design to appropriately inflate the sample size, accommodating finite numbers of clusters in the population when relevant. We use this framework to then discuss principled selection of survey design parameters in longitudinal surveillance programs. We apply this framework to design surveys to detect rises in malnutrition prevalence in nutrition surveillance programs in Kenya and South Sudan, accounting for clustering within villages. By combining historical information with data from previous surveys, we design surveys to detect spikes in the childhood malnutrition rate.

New tools for evaluating LQAS survey designs

Lot Quality Assurance Sampling (LQAS) surveys have become increasingly popular in global health care applications. Incorporating Bayesian ideas into LQAS survey design, such as using reasonable prior beliefs about the distribution of an indicator, can improve the selection of design parameters and decision rules. In this paper, a joint frequentist and Bayesian framework is proposed for evaluating LQAS classification accuracy and informing survey design parameters. Simple software tools are provided for calculating the positive and negative predictive value of a design with respect to an underlying coverage distribution and the selected design parameters. These tools are illustrated using a data example from two consecutive LQAS surveys measuring Oral Rehydration Solution (ORS) preparation. Using the survey tools, the dependence of classification accuracy on benchmark selection and the width of the ‘grey region’ are clarified in the context of ORS preparation across seven supervision areas. Following the completion of an LQAS survey, estimation of the distribution of coverage across areas facilitates quantifying classification accuracy and can help guide intervention decisions.

Estimation after classification using lot quality assurance sampling: corrections for curtailed sampling with application to evaluating polio vaccination campaigns

OBJECTIVES

To assess the bias incurred when curtailment of Lot Quality Assurance Sampling (LQAS) is ignored, to present unbiased estimators, to consider the impact of cluster sampling by simulation and to apply our method to published polio immunization data from Nigeria.

METHODS

We present estimators of coverage when using two kinds of curtailed LQAS strategies: semicurtailed and curtailed. We study the proposed estimators with independent and clustered data using three field-tested LQAS designs for assessing polio vaccination coverage, with samples of size 60 and decision rules of 9, 21 and 33, and compare them to biased maximum likelihood estimators. Lastly, we present estimates of polio vaccination coverage from previously published data in 20 local government authorities (LGAs) from five Nigerian states.

RESULTS

Simulations illustrate substantial bias if one ignores the curtailed sampling design. Proposed estimators show no bias. Clustering does not affect the bias of these estimators. Across simulations, standard errors show signs of inflation as clustering increases. Neither sampling strategy nor LQAS design influences estimates of polio vaccination coverage in 20 Nigerian LGAs. When coverage is low, semicurtailed LQAS strategies considerably reduces the sample size required to make a decision. Curtailed LQAS designs further reduce the sample size when coverage is high.

CONCLUSIONS

Results presented dispel the misconception that curtailed LQAS data are unsuitable for estimation. These findings augment the utility of LQAS as a tool for monitoring vaccination efforts by demonstrating that unbiased estimation using curtailed designs is not only possible but these designs also reduce the sample size.