An experimental evaluation of a stopping rule aimed at maximizing cost-quality trade-offs in surveys

Surveys face difficult choices in managing cost-error trade-offs. Stopping rules for surveys have been proposed as a method for managing these trade-offs. A stopping rule will limit effort on a select subset of cases to reduce costs with minimal harm to quality. Previously proposed stopping rules have focused on quality with an implicit assumption that all cases have the same cost. This assumption is unlikely to be true, particularly when some cases will require more effort and, therefore, more costs than others. We propose a new rule that looks at both predicted costs and quality. This rule is tested experimentally against another rule that focuses on stopping cases that are expected to be difficult to recruit. The experiment was conducted on the 2020 data collection of the Health and Retirement Study (HRS). We test both Bayesian and non-Bayesian (maximum-likelihood or ML) versions of the rule. The Bayesian version of the prediction models uses historical data to establish prior information. The Bayesian version led to higher-quality data for roughly the same cost, while the ML version led to small reductions in quality with larger reductions in cost compared to the control rule.

Influenza Vaccination Among Pregnant Women: Self-report Compared With Vaccination Data From Electronic Health Records, 2018-2020 Influenza Seasons

OBJECTIVES

Having accurate influenza vaccination coverage estimates can guide public health activities. The objectives of this study were to (1) validate the accuracy of electronic health record (EHR)-based influenza vaccination data among pregnant women compared with survey self-report and (2) assess whether survey respondents differed from survey nonrespondents by demographic characteristics and EHR-based vaccination status.

METHODS

This study was conducted in the Vaccine Safety Datalink, a network of 8 large medical care organizations in the United States. Using EHR data, we identified all women pregnant during the 2018-2019 or 2019-2020 influenza seasons. Surveys were conducted among samples of women who did and did not appear vaccinated for influenza according to EHR data. Separate surveys were conducted after each influenza season, and respondents reported their influenza vaccination status. Analyses accounted for the stratified design, sampling probability, and response probability.

RESULTS

The survey response rate was 50.5% (630 of 1247) for 2018-2019 and 41.2% (721 of 1748) for 2019-2020. In multivariable analyses combining both survey years, non-Hispanic Black pregnant women had 3.80 (95% CI, 2.13-6.74) times the adjusted odds of survey nonresponse; odds of nonresponse were also higher for Hispanic pregnant women and women who had not received (per EHR data) influenza vaccine during current or prior influenza seasons. The sensitivity, specificity, and positive predictive value of EHR documentation of influenza vaccination compared with self-report were ≥92% for both survey years combined. The negative predictive value of EHR-based influenza vaccine status was 80.5% (95% CI, 76.7%-84.0%).

CONCLUSIONS

EHR-based influenza vaccination data among pregnant women were generally concordant with self-report. New data sources and novel approaches to mitigating nonresponse bias may be needed to enhance influenza vaccination surveillance efforts.

Empirical Comparison of Imputation Methods for Multivariate Missing Data in Public Health

Sample estimates derived from data with missing values may be unreliable and may negatively impact the inferences that researchers make about the underlying population due to nonresponse bias. As a result, imputation is often preferred to listwise deletion in handling multivariate missing data. In this study, we compared three popular imputation methods: sequential multiple imputation, fractional hot-deck imputation, and generalized efficient regression-based imputation with latent processes for handling multivariate missingness under different missing patterns by conducting descriptive and regression analyses on the imputed data and seeing how the estimates differ from those generated from the full sample. Limited Monte Carlo simulation results by using the National Health Nutrition and Examination Survey and Behavioral Risk Factor Surveillance System are presented to demonstrate the effect of each imputation method on reducing bias and increasing efficiency for the parameter estimate of interest for that particular incomplete variable. Although these three methods did not always outperform listwise deletion in our simulated missing patterns, they improved many descriptive and regression estimates when used to impute all incomplete variables at once.

Successive Wave Analysis to Assess Nonresponse Bias in a Statewide Random Sample Testing Study for SARS-CoV-2

INTRODUCTION

Nonresponse bias occurs when participants in a study differ from eligible nonparticipants in ways that can distort study conclusions. The current study uses successive wave analysis, an established but underutilized approach, to assess nonresponse bias in a large-scale SARS-CoV-2 prevalence study. Such an approach makes use of reminders to induce participation among individuals. Based on the response continuum theory, those requiring several reminders to participate are more like nonrespondents than those who participate in a study upon first invitation, thus allowing for an examination of factors affecting participation.

METHODS

Study participants from the Indiana Population Prevalence SARS-CoV-2 Study were divided into 3 groups (eg, waves) based upon the number of reminders that were needed to induce participation. Independent variables were then used to determine whether key demographic characteristics as well as other variables hypothesized to influence study participation differed by wave using chi-square analyses. Specifically, we examined whether race, age, gender, education level, health status, tobacco behaviors, COVID-19-related symptoms, reasons for participating in the study, and SARS-CoV-2 positivity rates differed by wave.

RESULTS

Respondents included 3658 individuals, including 1495 in wave 1 (40.9%), 1246 in wave 2 (34.1%), and 917 in wave 3 (25%), for an overall participation rate of 23.6%. No significant differences in any examined variables were observed across waves, suggesting similar characteristics among those needing additional reminders compared with early participants.

CONCLUSIONS

Using established techniques, we found no evidence of nonresponse bias in a random sample with a relatively low response rate. A hypothetical additional wave of participants would be unlikely to change original study conclusions. Successive wave analysis is an effective and easy tool that can allow public health researchers to assess, and possibly adjust for, nonresponse in any epidemiological survey that uses reminders to encourage participation.

Improving inferences about private land conservation by accounting for incomplete reporting

Private lands provide key habitat for imperiled species and are core components of function protectected area networks; yet, their incorporation into national and regional conservation planning has been challenging. Identifying locations where private landowners are likely to participate in conservation initiatives can help avoid conflict and clarify trade-offs between ecological benefits and sociopolitical costs. Empirical, spatially explicit assessment of the factors associated with conservation on private land is an emerging tool for identifying future conservation opportunities. However, most data on private land conservation are voluntarily reported and incomplete, which complicates these assessments. We used a novel application of occupancy models to analyze the occurrence of conservation easements on private land. We compared multiple formulations of occupancy models with a logistic regression model to predict the locations of conservation easements based on a spatially explicit social-ecological systems framework. We combined a simulation experiment with a case study of easement data in Idaho and Montana (United States) to illustrate the utility of the occupancy framework for modeling conservation on private land. Occupancy models that explicitly accounted for variation in reporting produced estimates of predictors that were substantially less biased than estimates produced by logistic regression under all simulated conditions. Occupancy models produced estimates for the 6 predictors we evaluated in our case study that were larger in magnitude, but less certain than those produced by logistic regression. These results suggest that occupancy models result in qualitatively different inferences regarding the effects of predictors on conservation easement occurrence than logistic regression and highlight the importance of integrating variable and incomplete reporting of participation in empirical analysis of conservation initiatives. Failure to do so can lead to emphasizing the wrong social, institutional, and environmental factors that enable conservation and underestimating conservation opportunities in landscapes where social norms or institutional constraints inhibit reporting.

Differences in Nonresponse Bias and Victimization Reports Across Self-Administered Web-Based and Paper-and-Pencil Versions of a Campus Climate Survey

Using a standardized campus climate survey that was disseminated across three modes of administration (N = 5,137), this study assesses the nonresponse bias of two web-based versions to a self-administered paper-and-pencil version conducted at a Southeastern 4-year university. Significant differences emerged across all three modes of administration and victimization measures (bullying, sexual assault, rape, emotional abuse, and intimate partner violence [IPV]). Respondents were more likely to report victimization in the web-based surveys administered to online-only classes and via mass email compared to the paper survey. Policy implications, especially as it relates to survey administration, are discussed.

National Pharmacist Workforce Study (NPWS): Description of 2019 Survey Methods and Assessment of Nonresponse Bias

National Pharmacist Workforce Studies (NPWS) have been conducted in the U.S. every five years since 2000. This article describes the online survey methods used for the latest NPWS conducted in 2019 and provides an assessment for nonresponse bias. Three waves of emails containing a link to the online survey were sent to a random sample of about 96,000 pharmacists licensed in the United States. The survey asked about pharmacist employment, work activities, work–life balance, practice characteristics, pharmacist demographics and training. A total of 5467 usable responses were received, for a usable response rate of 5.8%. To assess for nonresponse bias, respondent characteristics were compared to the population of U.S. pharmacists and a benchmark, while a wave analysis compared early and late respondents. The pharmacist sample–population comparison and the benchmark comparison showed that the NPWS respondents had a higher percentage of female pharmacists and a lower proportion of young pharmacists compared to the population of U.S. pharmacists and the benchmark sample. In some contrast, the wave analysis showed that the early respondents had a higher percentage of males and older pharmacists compared to the late respondents. Both the wave analysis and the benchmark comparison showed that the NPWS respondents (and early respondents) had a lower percent of pharmacists with a PharmD degree than did the late respondents and the benchmark group. These differences should be considered when interpreting the findings from the 2019 NPWS.

Longitudinal Uses of the Population Assessment of Tobacco and Health Study

Objectives

The Population Assessment of Tobacco and Health (PATH) Study is a nationally representative study of the US population on tobacco use and its effects on health, with 3 waves of data collection between 2013 and 2016. Prior work described the methods of the first wave. In this paper, we describe the methods of the subsequent 2 waves and provide recommendations for how to conduct longitudinal analyses of PATH Study data.

Methods

We use standard survey quality metrics to evaluate the results of the follow-up waves of the PATH Study. The recommendations and examples of longitudinal and cross-sectional analyses of PATH Study data follow a design-based statistical inference framework.

Results

The quality metrics indicate that the PATH Study sample of approximately 40,000 continuing respondents remains representative of its target population. Depending on the intended analysis, different survey weights may be appropriate.

Conclusion

The PATH Study data are a valuable resource for regulatory scientists interested in longitudinal analysis of tobacco use and its effects on health. The availability of multiple sets of specialized survey weights enables researchers to target a wide range of tobacco-related analytic questions.

Methods for Improving Representativeness in a Web Survey on Sexual Assault Among College Students

Many colleges and universities conduct web-based campus climate surveys to understand the prevalence and nature of sexual assault among their students. When designing and fielding a web survey to measure a sensitive topic like sexual assault, methodological decisions, including the length of the field period and the use or amount of an incentive, can affect the representativeness of the respondent sample leading to biased or imprecise estimates. This study uses data from the Campus Climate Survey Validation Study (CCSVS) to assess how the interaction between field period length and survey incentive amount affects nonresponse, sample representativeness, and the precision of survey estimates. Research suggests that using robust incentives gives potential survey respondents a reason to complete the survey beyond their intrinsic motivation to do so. Likewise, extending the field period gives more time to people who may be less intrinsically motivated to complete the survey. Both serve to increase sample size and representativeness, minimize bias, and improve estimate precision. Schools, however, sometimes lack the time and/or resources for both a robust incentive and a lengthy field period, and this study examines the extent to which the potential negative impacts of not using one can be mitigated by the presence of the other. Findings indicate that target response rates can be achieved using a smaller incentive if the field period is lengthy but, even with a lengthy field period, the use of a smaller incentive can result in biased estimates due to a lack of representativeness. Conversely, when a robust incentive is used and weights are developed to adjust for nonresponse, a shorter field period will not have a significant impact on point estimates, but the estimates will be less precise due to fewer respondents participating in the survey.

Who provides open-ended comments? Assessing item nonresponse bias in Medicaid surveys of members and dentists

OBJECTIVES

This aim of this study was to examine factors associated with survey item nonresponse to open-ended items in mailed surveys.

METHODS

Data sources include two surveys conducted in Iowa in 2016 - one to a random sample of Medicaid dental program members and one to private practice dentists. Item nonresponse bias for open-ended comments was examined by comparing differences between commenters and noncommenters. Bivariate and logistic regression analyses examined differences based on demographic characteristics, attitudes, and experiences with the program, and survey mode.

RESULTS

Among members, respondents who were Black, older, unemployed, had a recent dental visit, rated the plan poorly, and completed the survey on paper were significantly more likely to provide comment. Among dentists, those who participated in the plan and those who completed the survey online were significantly more likely to provide comment.

CONCLUSIONS

Members and dentists with direct experiences with the Medicaid dental program were more likely to provide open-ended survey comments, whereas we found inconsistent results between members and dentists with regard to the impact of demographic characteristics, survey mode, and attitude toward the plan.

The effect of administration mode on CAHPS survey response rates and results: A comparison of mail and web-based approaches

OBJECTIVE

The objective of this study was to compare response rates, respondents' characteristics, and substantive results for CAHPS surveys administered using web and mail protocols.

DATA SOURCES

Patients who had one or more primary care visits in the preceding 6 months.

STUDY DESIGN/DATA COLLECTION METHODS

Patients for whom primary care practices had email addresses were randomized to one of four survey administration protocols: web via a portal invitation; web via an email invitation; combination of web and mail; and mail only. Another sample of patients without known email addresses was surveyed by mail. Samples of nonrespondents to the Internet and mail protocols were surveyed by telephone.

PRINCIPAL FINDINGS

Response rates to surveys administered using the Internet protocols were lower than for the surveys administered by mail (20 percent vs over 40 percent). However, characteristics of respondents and survey answers were very similar across protocols. Respondents without email addresses were older, less educated, and more likely to be male than those with email addresses, and there were a few differences in their responses. There was little evidence of nonresponse bias in either the mail or web protocols.

CONCLUSION

In this well-educated patient population, web protocols had lower response rates, but substantive results very similar to those from mail protocols.

Combining Participatory Influenza Surveillance with Modeling and Forecasting: Three Alternative Approaches

BACKGROUND

Influenza outbreaks affect millions of people every year and its surveillance is usually carried out in developed countries through a network of sentinel doctors who report the weekly number of Influenza-like Illness cases observed among the visited patients. Monitoring and forecasting the evolution of these outbreaks supports decision makers in designing effective interventions and allocating resources to mitigate their impact.

OBJECTIVE

Describe the existing participatory surveillance approaches that have been used for modeling and forecasting of the seasonal influenza epidemic, and how they can help strengthen real-time epidemic science and provide a more rigorous understanding of epidemic conditions.

METHODS

We describe three different participatory surveillance systems, WISDM (Widely Internet Sourced Distributed Monitoring), Influenzanet and Flu Near You (FNY), and show how modeling and simulation can be or has been combined with participatory disease surveillance to: i) measure the non-response bias in a participatory surveillance sample using WISDM; and ii) nowcast and forecast influenza activity in different parts of the world (using Influenzanet and Flu Near You).

RESULTS

WISDM-based results measure the participatory and sample bias for three epidemic metrics i.e. attack rate, peak infection rate, and time-to-peak, and find the participatory bias to be the largest component of the total bias. The Influenzanet platform shows that digital participatory surveillance data combined with a realistic data-driven epidemiological model can provide both short-term and long-term forecasts of epidemic intensities, and the ground truth data lie within the 95 percent confidence intervals for most weeks. The statistical accuracy of the ensemble forecasts increase as the season progresses. The Flu Near You platform shows that participatory surveillance data provide accurate short-term flu activity forecasts and influenza activity predictions. The correlation of the HealthMap Flu Trends estimates with the observed CDC ILI rates is 0.99 for 2013-2015. Additional data sources lead to an error reduction of about 40% when compared to the estimates of the model that only incorporates CDC historical information.

CONCLUSIONS

While the advantages of participatory surveillance, compared to traditional surveillance, include its timeliness, lower costs, and broader reach, it is limited by a lack of control over the characteristics of the population sample. Modeling and simulation can help overcome this limitation as well as provide real-time and long-term forecasting of influenza activity in data-poor parts of the world.

Understanding Response Rates to Surveys About Family Members' Psychological Symptoms After Patients' Critical Illness

CONTEXT

Achieving adequate response rates from family members of critically ill patients can be challenging, especially when assessing psychological symptoms.

OBJECTIVES

To identify factors associated with completion of surveys about psychological symptoms among family members of critically ill patients.

METHODS

Using data from a randomized trial of an intervention to improve communication between clinicians and families of critically ill patients, we examined patient-level and family-level predictors of the return of usable surveys at baseline, three months, and six months (n = 181, 171, and 155, respectively). Family-level predictors included baseline symptoms of psychological distress, decisional independence preference, and attachment style. We hypothesized that family with fewer symptoms of psychological distress, a preference for less decisional independence, and secure attachment style would be more likely to return questionnaires.

RESULTS

We identified several predictors of the return of usable questionnaires. Better self-assessed family member health status was associated with a higher likelihood and stronger agreement with a support-seeking attachment style with a lower likelihood, of obtaining usable baseline surveys. At three months, family-level predictors of return of usable surveys included having usable baseline surveys, status as the patient's legal next of kin, and stronger agreement with a secure attachment style. The only predictor of receipt of surveys at six months was the presence of usable surveys at three months.

CONCLUSION

We identified several predictors of the receipt of surveys assessing psychological symptoms in family of critically ill patients, including family member health status and attachment style. Using these characteristics to inform follow-up mailings and reminders may enhance response rates.

A Nonresponse Bias Analysis of the Health Information National Trends Survey (HINTS)

We conducted a nonresponse bias analysis of the Health Information National Trends Survey (HINTS) 4, Cycles 1 and 3, collected in 2011 and 2013, respectively, using three analysis methods: comparison of response rates for subgroups, comparison of estimates with weighting adjustments and external benchmarks, and level-of-effort analysis. Areas with higher concentrations of low socioeconomic status, higher concentrations of young households, and higher concentrations of minority and Hispanic populations had lower response rates. Estimates of health information seeking behavior were higher in HINTS compared to the National Health Interview Survey (NHIS). The HINTS estimate of doctors always explaining things in a way that the patient understands was not significantly different from the same estimate from the Medical Expenditure Panel Survey (MEPS); however, the HINTS estimate of health professionals always spending enough time with the patient was significantly lower than the same estimate from MEPS. A level-of-effort analysis found that those who respond later in the survey field period were less likely to have looked for information about health in the past 12 months, but found only small differences between early and late respondents for the majority of estimates examined. There is some evidence that estimates from HINTS could be biased toward finding higher levels of health information seeking.

Establishment of a cohort for deep phenotyping of the immune response to influenza vaccination among elderly individuals recruited from the general population

Elderly individuals have the highest burden of disease from influenza infection but also the lowest immune response to influenza vaccination. A better understanding of the host response to influenza vaccination in the elderly is therefore urgently needed. We conducted a biphasic prospective, population-based study from Dec. 2014 to May 2015 (pilot study) and Sept. 2015 to May 2016 (main study). Individuals 65-80 y of age were randomly selected from the residents' registration office in Hannover, Germany, for the pilot (n = 34) and main study (n = 200). The pilot study tested recruitment for study arms featuring 2, 4, or 5 visits/blood draws. The 5-visit (day 0, 1/3, 7, 21, 70 with respect to vaccination) study arm was selected for the main study. Both studies featured vaccination with Fluad™ (Novartis, Italy), a detailed medical history, a physical exam, recording of adverse events, completion of a questionnaire on common infections and an end-of-study questionnaire, and blood samples. Response rates in the pilot and main studies were 3.7% and 4.0%, respectively. Willingness to participate did not differ among the study arms (Fisher's exact test, p = 0.44). In both studies, there were no losses to follow-up. Compliance with study visits, blood sampling and completion of the questionnaires was very high (100%, >97%, 100%, respectively), as were participants' acceptance of and satisfaction with both phases of the study. The low response rates indicate the need for optimized recruitment strategies if the study population is to be representative of the general population. Nonetheless, the complex prospective study design proved to be highly feasible.

An ad hoc method for dual adjusting for measurement errors and nonresponse bias for estimating prevalence in survey data: Application to Iranian mental health survey on any illicit drug use

Purpose The prevalence estimates of binary variables in sample surveys are often subject to two systematic errors: measurement error and nonresponse bias. A multiple-bias analysis is essential to adjust for both biases. Methods In this paper, we linked the latent class log-linear and proxy pattern-mixture models to adjust jointly for measurement errors and nonresponse bias with missing not at random mechanism. These methods were employed to estimate the prevalence of any illicit drug use based on Iranian Mental Health Survey data. Results After jointly adjusting for measurement errors and nonresponse bias in this data, the prevalence (95% confidence interval) estimate of any illicit drug use changed from 3.41 (3.00, 3.81)% to 27.03 (9.02, 38.76)%, 27.42 (9.04, 38.91)%, and 27.18 (9.03, 38.82)% under "missing at random," "missing not at random," and an intermediate mode, respectively. Conclusions Under certain assumptions, a combination of the latent class log-linear and binary-outcome proxy pattern-mixture models can be used to jointly adjust for both measurement errors and nonresponse bias in the prevalence estimation of binary variables in surveys.

Estimating the Prevalence of Ovarian Cancer Symptoms in Women Aged 50 Years or Older: Problems and Possibilities

Diagnostic testing is recommended in women with "ovarian cancer symptoms." However, these symptoms are nonspecific. The ongoing Diagnosing Ovarian Cancer Early (DOVE) Study in Montreal, Quebec, Canada, provides diagnostic testing to women aged 50 years or older with symptoms lasting for more than 2 weeks and less than 1 year. The prevalence of ovarian cancer in DOVE is 10 times that of large screening trials, prompting us to estimate the prevalence of these symptoms in this population. We sent a questionnaire to 3,000 randomly sampled women in 2014-2015. Overall, 833 women responded; 81.5% reported at least 1 symptom, and 59.7% reported at least 1 symptom within the duration window specified in DOVE. We explored whether such high prevalence resulted from low survey response by applying inverse probability weighting to correct the estimates. Older women and those from deprived areas were less likely to respond, but only age was associated with symptom reporting. Prevalence was similar in early and late responders. Inverse probability weighting had a minimal impact on estimates, suggesting little evidence of nonresponse bias. This is the first study investigating symptoms that have proven to identify a subset of women with a high prevalence of ovarian cancer. However, the high frequency of symptoms warrants further refinements before symptom-triggered diagnostic testing can be implemented.

Timing the Mode Switch in a Sequential Mixed-Mode Survey: An Experimental Evaluation of the Impact on Final Response Rates, Key Estimates, and Costs

Mixed-mode surveys need to determine a number of design parameters that may have a strong influence on costs and errors. In a sequential mixed-mode design with web followed by telephone, one of these decisions is when to switch modes. The web mode is relatively inexpensive but produces lower response rates. The telephone mode complements the web mode in that it is relatively expensive but produces higher response rates. Among the potential negative consequences, delaying the switch from web to telephone may lead to lower response rates if the effectiveness of the prenotification contact materials is reduced by longer time lags, or if the additional e-mail reminders to complete the web survey annoy the sampled person. On the positive side, delaying the switch may decrease the costs of the survey. We evaluate these costs and errors by experimentally testing four different timings (1, 2, 3, or 4 weeks) for the mode switch in a web-telephone survey. This experiment was conducted on the fourth wave of a longitudinal study of the mental health of soldiers in the U.S. Army. We find that the different timings of the switch in the range of 1-4 weeks do not produce differences in final response rates or key estimates but longer delays before switching do lead to lower costs.

Health Estimates Using Survey Raked-Weighting Techniques in an Australian Population Health Surveillance System

A challenge for population health surveillance systems using telephone methodologies is to maintain representative estimates as response rates decrease. Raked weighting, rather than conventional poststratification methodologies, has been developed to improve representativeness of estimates produced from telephone-based surveillance systems by incorporating a wider range of sociodemographic variables using an iterative proportional fitting process. This study examines this alternative weighting methodology with the monthly South Australian population health surveillance system report of randomly selected people of all ages in 2013 (n = 7,193) using computer-assisted telephone interviewing. Poststratification weighting used age groups, sex, and area of residence. Raked weights included an additional 6 variables: dwelling status, number of people in household, country of birth, marital status, educational level, and highest employment status. Most prevalence estimates (e.g., diabetes and asthma) did not change when raked weights were applied. Estimates that changed by at least 2 percentage points (e.g., tobacco smoking and mental health conditions) were associated with socioeconomic circumstances, such as dwelling status, which were included in the raked-weighting methodology. Raking methodology has overcome, to some extent, nonresponse bias associated with the sampling methodology by incorporating lower socioeconomic groups and those who are routinely not participating in population surveys into the weighting formula.

What we don't know can hurt us: Nonresponse bias assessment in birth defects research

BACKGROUND

Nonresponse bias assessment is an important and underutilized tool in survey research to assess potential bias due to incomplete participation. This study illustrates a nonresponse bias sensitivity assessment using a survey on perceived barriers to care for children with orofacial clefts in North Carolina.

METHODS

Children born in North Carolina between 2001 and 2004 with an orofacial cleft were eligible for inclusion. Vital statistics data, including maternal and child characteristics, were available on all eligible subjects. Missing 'responses' from nonparticipants were imputed using assumptions based on the distribution of responses, survey method (mail or phone), and participant maternal demographics.

RESULTS

Overall, 245 of 475 subjects (51.6%) responded to either a mail or phone survey. Cost as a barrier to care was reported by 25.0% of participants. When stratified by survey type, 28.3% of mail respondents and 17.2% of phone respondents reported cost as a barrier. Under various assumptions, the bias-adjusted estimated prevalence of cost as barrier to care ranged from 16.1% to 30.0%. Maternal age, education, race, and marital status at time of birth were not associated with subjects reporting cost as a barrier.

CONCLUSION

As survey response rates continue to decline, the importance of assessing the potential impact of nonresponse bias has become more critical. Birth defects research is particularly conducive to nonresponse bias analysis, especially when birth defect registries and birth certificate records are used. Future birth defect studies which use population-based surveillance data and have incomplete participation could benefit from this type of nonresponse bias assessment. Birth Defects Research (Part A) 103:603-609, 2015. © 2015 Wiley Periodicals, Inc.

Surveying multiple health professional team members within institutional settings: an example from the nursing home industry

Quality improvement and cost containment initiatives in health care increasingly involve interdisciplinary teams of providers. To understand organizational functioning, information is often needed from multiple members of a leadership team since no one person may have sufficient knowledge of all aspects of the organization. To minimize survey burden, it is ideal to ask unique questions of each member of the leadership team in areas of their expertise. However, this risks substantial missing data if all eligible members of the organization do not respond to the survey. Nursing home administrators (NHA) and directors of nursing (DoN) play important roles in the leadership of long-term care facilities. Surveys were administered to NHAs and DoNs from a random, nationally representative sample of U.S. nursing homes about the impact of state policies, market forces, and organizational factors that impact provider performance and residents' outcomes. Responses were obtained from a total of 2,686 facilities (response rate [RR] = 66.6%) in which at least one individual completed the questionnaire and 1,693 facilities (RR = 42.0%) in which both providers participated. No evidence of nonresponse bias was detected. A high-quality representative sample of two providers in a long-term care facility can be obtained. It is possible to optimize data collection by obtaining unique information about the organization from each provider while minimizing the number of items asked of each individual. However, sufficient resources must be available for follow-up to nonresponders with particular attention paid to lower resourced, lower quality facilities caring for higher acuity residents in highly competitive nursing home markets.

Conclusions Regarding Cross-Group Differences in Happiness Depend on Difficulty of Reaching Respondents

A growing literature explores differences in subjective well-being across demographic groups, often relying on surveys with high nonresponse rates. By using the reported number of call attempts made to participants in the University of Michigan's Surveys of Consumers, we show that comparisons among easy-to-reach respondents differ from comparisons among hard-to-reach ones. Notably, easy-to-reach women are happier than easy-to-reach men, but hard-to-reach men are happier than hard-to-reach women, and conclusions of a survey could reverse with more attempted calls. Better alternatives to comparing group sample averages might include putting greater weight on hard-to-reach respondents or even extrapolating trends in responses.

Nonresponse bias in randomized controlled experiments in criminology: Putting the Queensland Community Engagement Trial (QCET) under a microscope

OBJECTIVES

The goal of this article is to examine whether or not the results of the Queensland Community Engagement Trial (QCET)-a randomized controlled trial that tested the impact of procedural justice policing on citizen attitudes toward police-were affected by different types of nonresponse bias.

METHOD

We use two methods (Cochrane and Elffers methods) to explore nonresponse bias: First, we assess the impact of the low response rate by examining the effects of nonresponse group differences between the experimental and control conditions and pooled variance under different scenarios. Second, we assess the degree to which item response rates are influenced by the control and experimental conditions.

RESULTS

Our analysis of the QCET data suggests that our substantive findings are not influenced by the low response rate in the trial. The results are robust even under extreme conditions, and statistical significance of the results would only be compromised in cases where the pooled variance was much larger for the nonresponse group and the difference between experimental and control conditions was greatly diminished. We also find that there were no biases in the item response rates across the experimental and control conditions.

CONCLUSION

RCTs that involve field survey responses-like QCET-are potentially compromised by low response rates and how item response rates might be influenced by the control or experimental conditions. Our results show that the QCET results were not sensitive to the overall low response rate across the experimental and control conditions and the item response rates were not significantly different across the experimental and control groups. Overall, our analysis suggests that the results of QCET are robust and any biases in the survey responses do not significantly influence the main experimental findings.

pharmacists in 2000: assessing nonresponse bias of a survey methodology

The first objective of this study was to assess the existence of nonresponse bias to a national survey of licensed pharmacists conducted in 2000. Three methods were used to assess nonresponse bias. The second objective of the study was to examine reasons why sampled licensed pharmacists did not respond to the national survey of licensed pharmacists. We used data from 2204 respondents to a national survey of pharmacists and from 521 respondents to a survey of nonrespondents to the national survey. We made comparisons between respondents for 5 variables: employment status, gender, age, highest academic degree, and year of initial licensure. Chi-square tests were used to examine differences in the 5 variables between respondents to the first mailing and second mailing of the survey, early and late respondents to the survey, and respondents to the survey and respondents to the nonrespondent survey. There were no significant differences between first mailing and second mailing respondents, but there were differences in each variable except year of licensure between early and late respondents. These differences likely were due to regional bias possibly related to differences in mailing times. There were differences between respondents and nonrespondents in terms of employment status and year of licensure. The main reasons for not responding to the survey were that it was too long or that it was too intrusive. Overall, the survey methodology resulted in a valid sample of licensed pharmacists. Nonresponse bias should be assessed by surveying nonrespondents. Future surveys of pharmacists should consider the length of the survey and the address where it is sent.