Measurement error in dietary assessment: an investigation using covariance structure models. Part I

Invited Commentary: The Contribution to the Field of Nutritional Epidemiology of the Landmark 1985 Publication by Willett et al

The semiquantitative food frequency questionnaire (FFQ) has been the primary source of dietary exposure data in epidemiology for decades. Although frequency instruments had been evaluated before the 1985 publication "Reproducibility and Validity of a Semiquantitative Food Frequency Questionnaire" by Willett et al. (Am J Epidemiol. 1985;122(1):51-65), that paper was the prototype for the development and validation of what was then a highly innovative method for collecting dietary data. This approach was adopted in nearly all subsequent cohort studies of diet and disease. The paper also catalyzed an extended scientific discourse regarding methods for validation, energy adjustment, and measurement error. It is now well established that data from FFQs and other self-reported dietary assessment instruments have both value and error and that this error should be considered in the analysis and interpretation of findings, including sensitivity analyses in which adjustment for measurement error is explored. Advances in technology make it feasible to consider collecting multiple granular short-term instruments such as recalls or records over time in addition to FFQs among all participants in large cohort studies; both provide valuable information. Without a doubt, the 1985 publication by Willett et al. provided the foundation that propelled the field of nutritional epidemiology forward, and it continues to be relevant today.

Indexes for Assessing Adherence to a Mediterranean Diet from Data Measured through Brief Questionnaires: Issues Raised from the Analysis of a Greek Population Study

Background: Adherence to a Mediterranean diet (MD) has been quantified through various indexes that rely on full-length questionnaires, but their application in data collected with brief questionnaires has not been systematically investigated. Objective: We aimed to evaluate the ability of the commonly used MD score (MDS) to classify individuals according to their adherence to an MD when applied to data collected with brief questionnaires. Methods: We assessed the diet of 200 participants from a Greek national health survey with the use of 2 instruments: 1) a validated, detailed food-frequency questionnaire (FFQ) estimating grams per day of intake of individual foods and 2) a selection of 19 questions from the Baseline Nutrition Credits4Health (BNC4H) electronic platform questionnaire that assessed servings with subjective serving sizes. We calculated the MDS_FFQ (referent) and MDS_BNC4H indexes from each questionnaire and estimated their correlation and the percentage of study participants who were ranked in an identical tertile in both indexes. We repeated the analyses for 2 additional indexes defined with criteria different from the MDS [Mediterranean Diet Index (MDI)_BNC4H and Mediterranean Diet Assessment Score (MEDAS)_ BNC4H]. Results: Spearman correlation coefficients for the MDS_FFQ were 0.31 with the MDS_BNC4H, 0.24 with the MDI_BNC4H, and 0.23 with the MEDAS_BNC4H. The proportion of participants ranked into the same adherence level as the referent MDS_FFQ was 50% for the MDS_BNC4H (weighted κ = 0.27) and lower for the other indexes. The use of medians as cutoffs (as in the MDS_FFQ) had low discriminative ability when applied to servings per day (MDS_BNC4H) in some MDS components, leading to comparability problems in the range of values between the 2 indexes. Conclusions: Our findings highlight certain issues that need to be considered when applying pre-existing MD indexes in settings with different dietary assessments. Given the widespread use of electronic platforms for dietary assessment, our results may offer further insight into designing brief, simplified questionnaires that aim to estimate MD adherence with easily quantifiable scores.

Methods to Assess Measurement Error in Questionnaires of Sedentary Behavior

Sedentary behavior has already been associated with mortality, cardiovascular disease, and cancer. Questionnaires are an affordable tool for measuring sedentary behavior in large epidemiological studies. Here, we introduce and evaluate two statistical methods for quantifying measurement error in questionnaires. Accurate estimates are needed for assessing questionnaire quality. The two methods would be applied to validation studies that measure a sedentary behavior by both questionnaire and accelerometer on multiple days. The first method fits a reduced model by assuming the accelerometer is without error, while the second method fits a more complete model that allows both measures to have error. Because accelerometers tend to be highly accurate, we show that ignoring the accelerometer's measurement error, can result in more accurate estimates of measurement error in some scenarios. In this manuscript, we derive asymptotic approximations for the Mean-Squared Error of the estimated parameters from both methods, evaluate their dependence on study design and behavior characteristics, and offer an R package so investigators can make an informed choice between the two methods. We demonstrate the difference between the two methods in a recent validation study comparing Previous Day Recalls (PDR) to an accelerometer-based ActivPal.

Addressing Current Criticism Regarding the Value of Self-Report Dietary Data

Recent reports have asserted that, because of energy underreporting, dietary self-report data suffer from measurement error so great that findings that rely on them are of no value. This commentary considers the amassed evidence that shows that self-report dietary intake data can successfully be used to inform dietary guidance and public health policy. Topics discussed include what is known and what can be done about the measurement error inherent in data collected by using self-report dietary assessment instruments and the extent and magnitude of underreporting energy compared with other nutrients and food groups. Also discussed is the overall impact of energy underreporting on dietary surveillance and nutritional epidemiology. In conclusion, 7 specific recommendations for collecting, analyzing, and interpreting self-report dietary data are provided: (1) continue to collect self-report dietary intake data because they contain valuable, rich, and critical information about foods and beverages consumed by populations that can be used to inform nutrition policy and assess diet-disease associations; (2) do not use self-reported energy intake as a measure of true energy intake; (3) do use self-reported energy intake for energy adjustment of other self-reported dietary constituents to improve risk estimation in studies of diet-health associations; (4) acknowledge the limitations of self-report dietary data and analyze and interpret them appropriately; (5) design studies and conduct analyses that allow adjustment for measurement error; (6) design new epidemiologic studies to collect dietary data from both short-term (recalls or food records) and long-term (food-frequency questionnaires) instruments on the entire study population to allow for maximizing the strengths of each instrument; and (7) continue to develop, evaluate, and further expand methods of dietary assessment, including dietary biomarkers and methods using new technologies.

Using surrogate biomarkers to improve measurement error models in nutritional epidemiology

Nutritional epidemiology relies largely on self-reported measures of dietary intake, errors in which give biased estimated diet–disease associations. Self-reported measurements come from questionnaires and food records. Unbiased biomarkers are scarce; however, surrogate biomarkers, which are correlated with intake but not unbiased, can also be useful. It is important to quantify and correct for the effects of measurement error on diet–disease associations. Challenges arise because there is no gold standard, and errors in self-reported measurements are correlated with true intake and each other. We describe an extended model for error in questionnaire, food record, and surrogate biomarker measurements. The focus is on estimating the degree of bias in estimated diet–disease associations due to measurement error. In particular, we propose using sensitivity analyses to assess the impact of changes in values of model parameters which are usually assumed fixed. The methods are motivated by and applied to measures of fruit and vegetable intake from questionnaires, 7-day diet diaries, and surrogate biomarker (plasma vitamin C) from over 25000 participants in the Norfolk cohort of the European Prospective Investigation into Cancer and Nutrition. Our results show that the estimated effects of error in self-reported measurements are highly sensitive to model assumptions, resulting in anything from a large attenuation to a small amplification in the diet–disease association. Commonly made assumptions could result in a large overcorrection for the effects of measurement error. Increased understanding of relationships between potential surrogate biomarkers and true dietary intake is essential for obtaining good estimates of the effects of measurement error in self-reported measurements on observed diet–disease associations. Copyright © 2013 John Wiley & Sons, Ltd.

[Estimate of corrected values and the effect of correction for measurement error in dietary data obtained by the Food Frequency Questionnaire for Adolescents (AFFQ)]

The scope of this study was to estimate calibrated values for dietary data obtained by the Food Frequency Questionnaire for Adolescents (FFQA) and illustrate the effect of this approach on food consumption data. The adolescents were assessed on two occasions, with an average interval of twelve months. In 2004, 393 adolescents participated, and 289 were then reassessed in 2005. Dietary data obtained by the FFQA were calibrated using the regression coefficients estimated from the average of two 24-hour recalls (24HR) of the subsample. The calibrated values were similar to the the 24HR reference measurement in the subsample. In 2004 and 2005 a significant difference was observed between the average consumption levels of the FFQA before and after calibration for all nutrients. With the use of calibrated data the proportion of schoolchildren who had fiber intake below the recommended level increased. Therefore, it is seen that calibrated data can be used to obtain adjusted associations due to reclassification of subjects within the predetermined categories.

Lifetime body size and reproductive factors: comparisons of data recorded prospectively with self reports in middle age

Background

Data on lifetime exposures are often self-reported in epidemiologic studies, sometimes many years after the relevant age. Validity of self-reported data is usually inferred from their agreement with measured values, but few studies directly quantify the likely effects of reporting errors in body size and reproductive history variables on estimates of disease-exposure associations.

Methods

The MRC National Survey of Health and Development (NSHD) and the Million Women Study (MWS) are UK population-based prospective cohorts. The NSHD recruited participants at birth in 1946 and has followed them at regular intervals since then, whereas the MWS recruited women in middle age. For 541 women who were participants in both studies, we used statistical measures of association and agreement to compare self-reported MWS data on body size throughout life and reproductive history, obtained in middle age, to NSHD data measured or reported close to the relevant ages. Likely attenuation of estimates of linear disease-exposure associations due to the combined effects of random and systematic errors was quantified using regression dilution ratios (RDRs).

Results

Data from the two studies were very strongly correlated for current height, weight and body mass index, and age at menopause (Pearson r = 0.91-0.95), strongly correlated for birth weight, parental heights, current waist and hip circumferences and waist-to-height ratio (r = 0.67-0.80), and moderately correlated for age at menarche and waist-to-hip ratio (r = 0.52-0.57). Self-reported categorical body size and clothes size data for various ages were moderately to strongly associated with anthropometry collected at the relevant times (Spearman correlations 0.51-0.79). Overall agreement between the studies was also good for most quantitative variables, although all exhibited both random and systematic reporting error. RDRs ranged from 0.66 to 0.86 for most variables (slight to moderate attenuation), except weight and body mass index (1.02 and 1.04, respectively; little or no attenuation), and age at menarche, birth weight and waist-to-hip ratio (0.44, 0.59 and 0.50, respectively; substantial attenuation).

Conclusions

This study provides some evidence that self-reported data on certain anthropometric and reproductive factors may be adequate for describing disease-exposure associations in large epidemiological studies, provided that the effects of reporting errors are quantified and the results are interpreted with caution.

Associations between dietary macronutrient intake and plasma lipids demonstrate criterion performance of the Multi-Ethnic Study of Atherosclerosis (MESA) food-frequency questionnaire

The validity of self-reported dietary intake is critical to the design and interpretation of diet-disease investigations. For many nutrients, there are no ideal methods to establish validity, given correlated error between reference and assessment tools, and constraints on time and resources available to perform such studies. Therefore, we quantified associations between macronutrient intakes and plasma HDL-cholesterol and TAG, relying on known associations between these factors to test the criterion validity of the FFQ used in the Multi-Ethnic Study of Atherosclerosis (MESA). Baseline dietary macronutrient intakes (derived from 120-item FFQ), and fasting plasma HDL and TAG were measured in 4510 MESA participants, aged 45-84 years. After adjusting for non-dietary factors known to affect plasma lipid concentrations, greater carbohydrate intake was associated with lower HDL and higher TAG (beta per 5-unit change in percentage energy intake from carbohydrate = - 5 (se 1) mg/l (P < 0.001) for HDL and 15 (se 6) mg/l (P = 0.008) for TAG), whereas higher energy intake from fat was associated with higher HDL and lower TAG (beta per 5-unit change in percentage energy from fat = 3.7 (se 2) mg/l (P = 0.01) for HDL and beta = 19 (se 7) mg/l (P = 0.004) for TAG). Associations of dietary carbohydrate and fat intakes with HDL and TAG concentrations were consistent with previous studies, demonstrating criterion validity of these dietary measures in the MESA.

Measurement error correction for nutritional exposures with correlated measurement error: use of the method of triads in a longitudinal setting

Nutritional exposures are often measured with considerable error in commonly used surrogate instruments such as the food frequency questionnaire (FFQ) (denoted by Q(i) for the ith subject). The error can be both systematic and random. The diet record (DR) denoted by R(i) for the ith subject is considered an alloyed gold standard. However, some authors have reported both systematic and random errors with this instrument as well.One goal in measurement error research is to estimate the regression coefficient of T(i) (true intake for the ith subject) on Q(i) denoted by lambda(TQ). If the systematic errors in Q(i) and R(i) (denoted by q(i) and r(i)) are uncorrelated, then one can obtain an unbiased estimate of lambda(TQ) by lambda(RQ) obtained by regressing R(i) on Q(i). However, if Corr(q(i), r(i))>0, then lambda(RQ)>lambda(TQ).In this paper, we propose a method for indirectly estimating lambda(TQ) even in the presence of correlated systematic error based on a longitudinal design where Q(i) (surrogate measure of dietary intake), R(i) (a reference measure of dietary intake), and M(i) (a biomarker) are available on the same subjects at 2 time points. In addition, between-person variation in mean levels of M(i) among people with the same dietary intake is also accounted for. The methodology is illustrated for dietary vitamin C intake based on longitudinal data from 323 subjects in the European Prospective Investigation of Cancer (EPIC)-Norfolk study who provided two measures of dietary vitamin C intake from the FFQ (Q(i)) and a 7-day DR (R(i)) and plasma vitamin C (M(i)) 4 years apart.

Fat, protein, and meat consumption and renal cell cancer risk: a pooled analysis of 13 prospective studies

BACKGROUND

Results of several case-control studies suggest that high consumption of meat (all meat, red meat, or processed meat) is associated with an increased risk of renal cell cancer, but only a few prospective studies have examined the associations of intakes of meat, fat, and protein with renal cell cancer.

METHODS

We conducted a pooled analysis of 13 prospective studies that included 530,469 women and 244,483 men and had follow-up times of up to 7-20 years to examine associations between meat, fat, and protein intakes and the risk of renal cell cancer. All participants had completed a validated food frequency questionnaire at study entry. Using the primary data from each study, we calculated the study-specific relative risks (RRs) for renal cell cancer by using Cox proportional hazards models and then pooled these RRs by using a random-effects model. All statistical tests were two-sided.

RESULTS

A total of 1,478 incident cases of renal cell cancer were identified (709 in women and 769 in men). We observed statistically significant positive associations or trends in pooled age-adjusted models for intakes of total fat, saturated fat, monounsaturated fat, polyunsaturated fat, cholesterol, total protein, and animal protein. However, these associations were attenuated and no longer statistically significant after adjusting for body mass index, fruit and vegetable intake, and alcohol intake. For example, the pooled age-adjusted RR of renal cell cancer for the highest vs the lowest quintile of intake for total fat was 1.30 (95% confidence interval [CI] = 1.08 to 1.56; P(trend) = .001) and for total protein was 1.17 (95% CI = 0.99 to 1.38; P(trend) = .02). By comparison, the pooled multivariable RR for the highest vs the lowest quintile of total fat intake was 1.10 (95% CI = 0.92 to 1.32; P(trend) = .31) and of total protein intake was 1.06 (95% CI = 0.89 to 1.26; P(trend) = .37). Intakes of red meat, processed meat, poultry, or seafood were not associated with the risk of renal cell cancer.

CONCLUSIONS

Intakes of fat and protein or their subtypes, red meat, processed meat, poultry, and seafood are not associated with risk of renal cell cancer.

The evaluation of the diet/disease relation in the EPIC study: considerations for the calibration and the disease models

BACKGROUND

International multicentre studies on diet and cancer are relatively new in epidemiological research. They offer a series of challenging methodological issues for the evaluation of the association between dietary exposure and disease outcomes, which can both be quite heterogeneous across different geographical regions. This requires considerable work to standardize dietary measurements at the food and the nutrient levels.

METHODS

Within the European Prospective Investigation into Cancer and Nutrition (EPIC), a calibration study was set up to express individual dietary intakes according to the same reference scale. A linear regression calibration model was used to correct the association between diet and disease for measurement errors in dietary exposures. In the present work, we describe an approach for analysing the EPIC data, using as an example the evaluation of the association between fish intake and colorectal cancer incidence.

RESULTS

Sex- and country-specific attenuation factors ranged from 0.083 to 0.784, with values overall higher for men compared with women. Hazard ratio estimates of colorectal cancer for a 10 g/day increase in fish intake were 0.97 [95% confidence interval (CI): 0.95-0.99] and 0.93 (0.88-0.98), before and after calibration, respectively.

CONCLUSIONS

In a multicentre study, the diet/disease association can be evaluated by exploiting the whole variability of intake over the entire study. Calibration may reduce between-centre heterogeneity in the diet-disease relationship caused by differential impact of measurement errors across cohorts.

Classical latent variable models for medical research

Latent variable models are commonly used in medical statistics, although often not referred to under this name. In this paper we describe classical latent variable models such as factor analysis, item response theory, latent class models and structural equation models. Their usefulness in medical research is demonstrated using real data. Examples include measurement of forced expiratory flow, measurement of physical disability, diagnosis of myocardial infarction and modelling the determinants of clients' satisfaction with counsellors' interviews.

Performance of a food-frequency questionnaire in the US NIH-AARP (National Institutes of Health-American Association of Retired Persons) Diet and Health Study

OBJECTIVE

We evaluated the performance of the food-frequency questionnaire (FFQ) administered to participants in the US NIH-AARP (National Institutes of Health-American Association of Retired Persons) Diet and Health Study, a cohort of 566 404 persons living in the USA and aged 50-71 years at baseline in 1995.

DESIGN

The 124-item FFQ was evaluated within a measurement error model using two non-consecutive 24-hour dietary recalls (24HRs) as the reference.

SETTING

Participants were from six states (California, Florida, Pennsylvania, New Jersey, North Carolina and Louisiana) and two metropolitan areas (Atlanta, Georgia and Detroit, Michigan).

SUBJECTS

A subgroup of the cohort consisting of 2053 individuals.

RESULTS

For the 26 nutrient constituents examined, estimated correlations with true intake (not energy-adjusted) ranged from 0.22 to 0.67, and attenuation factors ranged from 0.15 to 0.49. When adjusted for reported energy intake, performance improved; estimated correlations with true intake ranged from 0.36 to 0.76, and attenuation factors ranged from 0.24 to 0.68. These results compare favourably with those from other large prospective studies. However, previous biomarker-based studies suggest that, due to correlation of errors in FFQs and self-report reference instruments such as the 24HR, the correlations and attenuation factors observed in most calibration studies, including ours, tend to overestimate FFQ performance.

CONCLUSION

The performance of the FFQ in the NIH-AARP Diet and Health Study, in conjunction with the study's large sample size and wide range of dietary intake, is likely to allow detection of moderate (> or =1.8) relative risks between many energy-adjusted nutrients and common cancers.

Alcohol Intake and Renal Cell Cancer in a Pooled Analysis of 12 Prospective Studies

BACKGROUND

The association between alcohol intake and risk of renal cell cancer has been inconsistent in case-control studies. An inverse association between alcohol intake and risk of renal cell cancer has been suggested in a few prospective studies, but each of these studies included a small number of cases.

METHODS

We performed a pooled analysis of 12 prospective studies that included 530,469 women and 229,575 men with maximum follow-up times of 7-20 years. All participants had completed a validated food-frequency questionnaire at baseline. Using the primary data from each study, the study-specific relative risks (RRs) for renal cell cancer were calculated using Cox proportional hazards models and then pooled using a random-effects model. All statistical tests were two-sided.

RESULTS

A total of 1430 (711 women and 719 men) cases of incident renal cell cancer were identified. The study-standardized incidence rates of renal cell cancer were 23 per 100,000 person-years among nondrinkers and 15 per 100,000 person-years among those who drank 15 g/day or more of alcohol. Compared with nondrinking, alcohol consumption (> or = 15 g/day, equivalent to slightly more than one alcoholic drink per day) was associated with a decreased risk of renal cell cancer (pooled multivariable RR = 0.72, 95% confidence interval = 0.60 to 0.86; P(trend)<.001); statistically significant inverse trends with increasing intake were seen in both women and men. No difference by sex was observed (P(heterogeneity) = .89). Associations between alcohol intake and renal cell cancer were not statistically different across alcoholic beverage type (beer versus wine versus liquor) (P = .40).

CONCLUSION

Moderate alcohol consumption was associated with a lower risk of renal cell cancer among both women and men in this pooled analysis.

Validity and systematic error in measuring carotenoid consumption with dietary self-report instruments

Vegetables and fruits are rich in carotenoids, a group of compounds thought to protect against cancer. Studies of diet-disease associations need valid and reliable instruments for measuring dietary intake. The authors present a measurement error model to estimate the validity (defined as correlation between self-reported intake and "true" intake), systematic error, and reliability of two self-report dietary assessment methods. Carotenoid exposure is measured by repeated 24-hour recalls, a food frequency questionnaire (FFQ), and a plasma marker. The model is applied to 1,013 participants assigned between 1995 and 2000 to the nonintervention arm of the Women's Healthy Eating and Living Study, a randomized trial assessing the impact of a low-fat, high-vegetable/fruit/fiber diet on preventing new breast cancer events. Diagnostics including graphs are used to assess the goodness of fit. The validity of the instruments was 0.44 for the 24-hour recalls and 0.39 for the FFQ. Systematic error accounted for over 22% and 50% of measurement error variance for the 24-hour recalls and FFQ, respectively. The use of either self-report method alone in diet-disease studies could lead to substantial bias and error. Multiple methods of dietary assessment may provide more accurate estimates of true dietary intake.

Beyond logistic regression: structural equations modelling for binary variables and its application to investigating unobserved confounders

Background

Structural equation modelling (SEM) has been increasingly used in medical statistics for solving a system of related regression equations. However, a great obstacle for its wider use has been its difficulty in handling categorical variables within the framework of generalised linear models.

Methods

A large data set with a known structure among two related outcomes and three independent variables was generated to investigate the use of Yule's transformation of odds ratio (OR) into Q-metric by (OR-1)/(OR+1) to approximate Pearson's correlation coefficients between binary variables whose covariance structure can be further analysed by SEM. Percent of correctly classified events and non-events was compared with the classification obtained by logistic regression. The performance of SEM based on Q-metric was also checked on a small (N = 100) random sample of the data generated and on a real data set.

Results

SEM successfully recovered the generated model structure. SEM of real data suggested a significant influence of a latent confounding variable which would have not been detectable by standard logistic regression. SEM classification performance was broadly similar to that of the logistic regression.

Conclusion

The analysis of binary data can be greatly enhanced by Yule's transformation of odds ratios into estimated correlation matrix that can be further analysed by SEM. The interpretation of results is aided by expressing them as odds ratios which are the most frequently used measure of effect in medical statistics.

Fiber intake and incidence of colorectal cancer among 76,947 women and 47,279 men

Prospective cohort studies have consistently found no important link between fiber intake and risk of colorectal cancer. The recent large, prospective European Prospective Investigation into Cancer and Nutrition has challenged this paradigm by suggesting significant protection by high fiber intake. We prospectively investigated the association of fiber intake with the incidence of colon and rectal cancers in two large cohorts: the Nurses' Health Study (76,947 women) and the Health Professionals Follow-up Study (47,279 men). Diet was assessed repeatedly in 1984, 1986, 1990, and 1994 among women and in 1986, 1990, and 1994 among men. The incidence of cancer of the colon and rectum was ascertained up to the year 2000. Relative risk estimates were calculated using a Cox proportional hazards model simultaneously controlling for potential confounding variables. During follow-up including 1.8 million person-years and 1,596 cases of colorectal cancer, we found little association with fiber intake after controlling for confounding variables. The hazard ratio for a 5-g/d increase in fiber intake was 0.91 (95% confidence interval, 0.87-0.95) after adjusting for covariates used in the European Prospective Investigation into Cancer and Nutrition study and 0.99 (95% confidence interval, 0.95-1.04) after adjusting for additional confounding variables. Our data from two large prospective cohorts with long follow-up and repeated assessment of fiber intake and of a large number of potential confounding variables do not indicate an important association between fiber intake and colorectal cancer but reveal considerable confounding by other dietary and lifestyle factors.

On the Importance of Using Multiple Methods of Dietary Assessment

BACKGROUND

Plasma carotenoid concentrations reflect intake of vegetables and fruits, the major food sources of these compounds. This study compared the ability of 2 measures of dietary intake (24-hour diet recalls and food frequency questionnaires [FFQs]) to corroborate plasma carotenoid concentrations in a subset of women participating in a diet intervention trial.

METHODS

Plasma carotenoid concentrations and dietary intakes, estimated from 24-hour diet recalls and FFQs, were examined at baseline and 1 year later in a subset of 395 study participants (197 intervention and 198 comparison group). We used longitudinal models to examine associations between estimated intakes and plasma carotenoid concentrations. These analyses were stratified by study group and adjusted for body mass index (BMI), plasma cholesterol concentration, and total energy intake. We conducted simulations to compare mean-squared errors of prediction of each assessment method.

RESULTS

In mixed-effects models, the estimated carotenoid intakes from both dietary assessment methods were strongly associated with plasma concentrations of alpha-carotene, beta-carotene, and lutein. Furthermore, modeling the 2 sources of intake information as joint predictors reduced the prediction error.

CONCLUSION

These findings underscore the importance of using multiple measures of dietary assessment in studies examining diet-disease associations.

Correlations between estimated and true dietary intakes

PURPOSE

It is unclear how well questionnaire or so-called reference methods of dietary assessment correlate with true dietary intake. We develop a method to estimate such correlations.

METHODS

An error model is described that uses data from a food frequency questionnaire (Q), a reference method (R), and a biological marker (M). The model does not assume the classical error model for either R or M, or that the correlation between errors in the questionnaire and reference data is zero. Credible intervals can be placed about correlations between R, Q, M and true dietary data (T), also about the correlations between errors in reference and questionnaire data.

RESULTS

Application of this model to a validation data set mainly found correlations in the range 0.4 to 0.8, and that correlations (R,T) generally exceeded correlations (Q,T), providing evidence that R is more valid than Q. Estimated correlations between errors in R and Q were often far from zero suggesting that regression calibration to imperfect reference data is problematic unless these error correlations can be estimated.

CONCLUSION

A biological marker in addition to dietary data, allows calculation of correlations between estimated and true dietary intakes under reasonable assumptions about errors. However, sensitivity analyses are necessary on one variable.

Improved estimates of floating absolute risk

Floating absolute risks are an alternative way of presenting relative risk estimates for polychotomous risk factors. Instead of choosing one level of the risk factor as a reference category, each level is assigned a 'floated' variance which describes the uncertainty in risk without reference to another level. In this paper, a method for estimating the floated variances is presented that improves on the previously proposed 'heuristic' method. The estimates may be calculated iteratively with a simple algorithm. A benchmark for validating the floated variance estimates is also proposed and an interpretation of floating confidence intervals is given.

A comparison of a food frequency questionnaire with a 24-hour recall for use in an epidemiological cohort study: results from the biomarker-based Observing Protein and Energy Nutrition (OPEN) study

BACKGROUND

Most large cohort studies have used a food frequency questionnaire (FFQ) for assessing dietary intake. Several biomarker studies, however, have cast doubt on whether the FFQ has sufficient precision to allow detection of moderate but important diet-disease associations. We use data from the Observing Protein and Energy Nutrition (OPEN) study to compare the performance of a FFQ with that of a 24-hour recall (24HR).

METHODS

The OPEN study included 484 healthy volunteer participants (261 men, 223 women) from Montgomery County, Maryland, aged 40-69. Each participant was asked to complete a FFQ and 24HR on two occasions 3 months apart, and a doubly labelled water (DLW) assessment and two 24-hour urine collections during the 2 weeks after the first FFQ and 24HR assessment. For both the FFQ and 24HR and for both men and women, we calculated attenuation factors for absolute energy, absolute protein, and protein density.

RESULTS

For absolute energy and protein, a single FFQ's attenuation factor is 0.04-0.16. Repeat administrations lead to little improvement (0.08-0.19). Attenuation factors for a single 24HR are 0.10-0.20, but four repeats would yield attenuations of 0.20-0.37. For protein density a single FFQ has an attenuation of 0.3-0.4; for a single 24HR the attenuation factor is 0.15-0.25 but would increase to 0.35-0.50 with four repeats.

CONCLUSIONS

Because of severe attenuation, the FFQ cannot be recommended as an instrument for evaluating relations between absolute intake of energy or protein and disease. Although this attenuation is lessened in analyses of energy-adjusted protein, it remains substantial for both FFQ and multiple 24HR. The utility of either of these instruments for detecting important but moderate relative risks (between 1.5 and 2.0), even for energy-adjusted dietary factors, is questionable.

Markers of the validity of reported energy intake

Energy intake (EI) is the foundation of the diet, because all other nutrients must be provided within the quantity of food needed to fulfill the energy requirement. Thus if total EI is underestimated, it is probable that the intakes of other nutrients are also underestimated. Under conditions of weight stability, EI equals energy expenditure (EE). Because at the group level weight may be regarded as stable in the timescale of a dietary assessment, the validity of reported EI can be evaluated by comparing it with either measured EE or an estimate of the energy requirement of the population. This paper provides the first comprehensive review of studies in which EI was reported and EE was measured using the doubly labeled water technique. These conclusively demonstrate widespread bias to the underestimation of EI. Because energy requirements of populations or individuals can be conveniently expressed as multiples of the basal metabolic rate (BMR), EE:BMR, reported EI may also be expressed as EI:BMR for comparison. Values of EI:BMR falling below the 95% confidence limit of agreement between these two measures signify the presence of underreporting. A formula for calculating the lower 95% confidence limit was proposed by Goldberg et al. (the Goldberg cutoff). It has been used by numerous authors to identify individual underreporters in different dietary databases to explore the variables associated with underreporting. These studies are also comprehensively reviewed. They explore the characteristics of underreporters and the biases in estimating nutrient intake and in describing meal patterns associated with underreporting. This review also examines some of the problems for the interpretation of data introduced by underreporting and particularly by variable underreporting across subjects. Future directions for research are identified.

Uses and limitations of statistical accounting for random error correlations, in the validation of dietary questionnaire assessments

OBJECTIVE

To examine statistical models that account for correlation between random errors of different dietary assessment methods, in dietary validation studies.

SETTING

In nutritional epidemiology, sub-studies on the accuracy of the dietary questionnaire measurements are used to correct for biases in relative risk estimates induced by dietary assessment errors. Generally, such validation studies are based on the comparison of questionnaire measurements (Q) with food consumption records or 24-hour diet recalls (R). In recent years, the statistical analysis of such studies has been formalized more in terms of statistical models. This made the need of crucial model assumptions more explicit. One key assumption is that random errors must be uncorrelated between measurements Q and R, as well as between replicate measurements R1 and R2 within the same individual. These assumptions may not hold in practice, however. Therefore, more complex statistical models have been proposed to validate measurements Q by simultaneous comparisons with measurements R plus a biomarker M, accounting for correlations between the random errors of Q and R.

CONCLUSIONS

The more complex models accounting for random error correlations may work only for validation studies that include markers of diet based on physiological knowledge about the quantitative recovery, e.g. in urine, of specific elements such as nitrogen or potassium, or stable isotopes administered to the study subjects (e.g. the doubly labelled water method for assessment of energy expenditure). This type of marker, however, eliminates the problem of correlation of random errors between Q and R by simply taking the place of R, thus rendering complex statistical models unnecessary.

Bias and heteroscedastic memory error in self-reported health behavior: an investigation using covariance structure analysis

BACKGROUND

Frequent use of self-reports for investigating recent and past behavior in medical research requires statistical techniques capable of analyzing complex sources of bias associated with this methodology. In particular, although decreasing accuracy of recalling more distant past events is commonplace, the bias due to differential in memory errors resulting from it has rarely been modeled statistically.

METHODS

Covariance structure analysis was used to estimate the recall error of self-reported number of sexual partners for past periods of varying duration and its implication for the bias.

RESULTS

Results indicated increasing levels of inaccuracy for reports about more distant past. Considerable positive bias was found for a small fraction of respondents who reported ten or more partners in the last year, last two years and last five years. This is consistent with the effect of heteroscedastic random error where the majority of partners had been acquired in the more distant past and therefore were recalled less accurately than the partners acquired more recently to the time of interviewing.

CONCLUSIONS

Memory errors of this type depend on the salience of the events recalled and are likely to be present in many areas of health research based on self-reported behavior.

Development, validation and utilisation of food-frequency questionnaires - a review

OBJECTIVE

The purpose of this review is to provide guidance on the development, validation and use of food-frequency questionnaires (FFQs) for different study designs. It does not include any recommendations about the most appropriate method for dietary assessment (e.g. food-frequency questionnaire versus weighed record).

METHODS

A comprehensive search of electronic databases was carried out for publications from 1980 to 1999. Findings from the review were then commented upon and added to by a group of international experts.

RESULTS

Recommendations have been developed to aid in the design, validation and use of FFQs. Specific details of each of these areas are discussed in the text.

CONCLUSIONS

FFQs are being used in a variety of ways and different study designs. There is no gold standard for directly assessing the validity of FFQs. Nevertheless, the outcome of this review should help those wishing to develop or adapt an FFQ to validate it for its intended use.

Food frequency questionnaire and a screening test

We assessed the accuracy of a 141-item food frequency questionnaire as a screening test to detect high or low consumption of nutrients associated with cancer. Fifty-five men and 58 women participating in two population-based cohort studies in Miyagi, Japan, provided four three-day diet records over a one-year period and subsequently completed the questionnaire twice with a one-year interval. Pearson correlation coefficients between 17 nutrients measured by the diet records and the first questionnaire ranged from 0.24 to 0.85 (median 0.43), and those between the two questionnaires ranged from 0.47 to 0.91 (median 0.68). The sensitivity and specificity of the questionnaire for detecting high-alcohol, high-fat, low-calcium, and low-ascorbic acid consumers were 86.7% and 96.7%, 50.0% and 85.7%, 48.8% and 76.4%, and 61.9% and 70.0%, respectively. Receiver operating characteristic curves indicated comparable performance of the questionnaire and a three-day diet record, regarded as another screening test. The questionnaire performed poorly for other nutrients. The results indicate that our questionnaire is reasonably reproducible, comparable with the diet records, and useful as a screening test to detect high or low consumers of several nutrients associated with cancer for subsequent enrollment in dietary intervention trials or dietary counseling.

Use of biological markers to validate self-reported dietary intake in a random sample of the European Prospective Investigation into Cancer United Kingdom Norfolk cohort

BACKGROUND

The validity of dietary assessment methods should be established before diet-disease associations are reported.

OBJECTIVE

Our objective was to validate a 7-d food diary and a food-frequency questionnaire (FFQ) against independent biomarkers of intake in urine (nitrogen, potassium, and sodium) and blood (plasma ascorbic acid).

DESIGN

A total of 146 healthy middle-aged men and women were recruited from the European Prospective Investigation into Cancer UK Norfolk cohort, a free-living cohort of approximately 25000 persons. Over a 9-mo period, urinary nitrogen, potassium, and sodium were estimated from 2-6 complete 24-h urine collections in 134 subjects and plasma ascorbic acid was estimated from 2-3 fasting blood samples in 118 subjects. Subjects completed 2 FFQs and two 7-d food diaries.

RESULTS

In men and women combined, correlations between 24-h urinary nitrogen excretion and dietary intake from the 7-d food diary were high (r = 0.57-0.67) compared with those for the FFQ (r = 0.21-0.29). Similarly, correlations between urinary potassium and dietary potassium were higher for the 7-d food diary (r = 0.51-0.55) than for the FFQ (r = 0.32-0.34). There was no overall difference in correlations between plasma ascorbic acid and dietary vitamin C between the 7-d food diary (r = 0.40-0.52) and the FFQ (r = 0.44-0.45).

CONCLUSIONS

These data indicate that, despite increased subject burden, the 7-d food diary provided a better estimate of nitrogen and potassium intakes than did the FFQ in this study population. However, with respect to plasma ascorbic acid, both the FFQ and 7-d food diary provided a similar ranking of subjects according to vitamin C intake.

Empirical evidence of correlated biases in dietary assessment instruments and its implications

Multiple-day food records or 24-hour recalls are currently used as "reference" instruments to calibrate food frequency questionnaires (FFQs) and to adjust findings from nutritional epidemiologic studies for measurement error. The common adjustment is based on the critical requirements that errors in the reference instrument be independent of those in the FFQ and of true intake. When data on urinary nitrogen level, a valid reference biomarker for nitrogen intake, are used, evidence suggests that a dietary report reference instrument does not meet these requirements. In this paper, the authors introduce a new model that includes, for both the FFQ and the dietary report reference instrument, group-specific biases related to true intake and correlated person-specific biases. Data were obtained from a dietary assessment validation study carried out among 160 women at the Dunn Clinical Nutrition Center, Cambridge, United Kingdom, in 1988-1990. Using the biomarker measurements and dietary report measurements from this study, the authors compare the new model with alternative measurement error models proposed in the literature and demonstrate that it provides the best fit to the data. The new model suggests that, for these data, measurement error in the FFQ could lead to a 51% greater attenuation of true nutrient effect and the need for a 2.3 times larger study than would be estimated by the standard approach. The implications of the results for the ability of FFQ-based epidemiologic studies to detect important diet-disease associations are discussed.

5 a day fruit and vegetable intervention improves consumption in a low income population

OBJECTIVE

This study evaluated the Michigan Farmers' Market Nutrition Program in one Michigan county to determine its effect on fruit and vegetable consumption behavior.

SUBJECTS/SETTING

Subjects were selected from WIC and Community Action Agency populations: 564 low income women completed the pretest; 455 completed the posttest. Attrition rate was 19.3%.

INTERVENTION

Subjects were assigned to one of 4 interventions: education about the use, storage and nutritional value of fruits and vegetables, distribution of farmers' market coupons, both education and coupons, or no intervention.

DESIGN

Education-only and coupon and education groups were randomly assigned; clinic appointment timing determined assignment to no-intervention and coupon-only groups.

MAIN OUTCOME MEASURES

A self-administered questionnaire before and after intervention measured attitudes about fruit and vegetable consumption and intake of fruits and vegetables. WIC records documented redemption of coupons.

STATISTICAL ANALYSES

Data analysis included 2-way multivariate analysis of covariance, univariate analysis of covariance, logistic regression, and covariance structure modeling.

RESULTS

Both the education interventions and the coupon interventions had positive effects. Coupons had a direct effect on increasing fruit and vegetable consumption behavior but no effect on attitudes. Education had a direct effect on attitudes and seemed to exert an effect on consumption behavior through attitudes. The maximum impact of the intervention was achieved through a combination of education and coupons.

APPLICATIONS

This study demonstrated that a low-income population may be more likely to increase its fruit and vegetable consumption behavior when incentives such as coupons improve affordability.

Biased over- or under-reporting is characteristic of individuals whether over time or by different assessment methods

Seven studies with repeated measurements of energy intake and/or nitrogen intake were examined to determine whether misreporting is characteristic of some persons or occurs randomly. Four of the studies were validated by doubly labeled water measurements of energy expenditure. Reporting validity was expressed as the ratio of energy intake to energy expenditure. Ratios were consistently below the expected value of 1.0 for some subjects and consistently above 1.0 for others, indicating characteristic reporting validity within subjects. Two year-long studies provided 4 to 12 measurements and a total number of days sufficient to measure individual habitual intake. Subjects mean energy intake to basal metabolic rate (BMR) ratios were < 1.35 in 45% and 47% and < 1.35 at every measurement in 25% of subjects. This indicated persistent underreporting over time, because 1.35 x BMR is the minimum energy expenditure compatible with a normally active lifestyle. Three of the studies used more than 1 assessment method (validated by doubly labeled water and/or urinary nitrogen excretion). There was a tendency for persons determined to be underreporters by 1 method to be also underreporters when tested by other methods. We conclude that biased over- or underreporting is characteristic of some persons. Thus, repeat measurements do not necessarily provide valid measures of individual intake, extreme intakes may reflect under- and overreporting rather than true low or high intakes, and subjects most prone to reporting bias may be repeatedly misclassified in quantiles of the distribution. This presents a challenge to dietitians nutritionists, and statisticians both for the design of surveys and the handling of flawed data.

Calibration of the dietary questionnaire for a multiethnic cohort in Hawaii and Los Angeles

The performance of the dietary questionnaire used in a multiethnic cohort study in Hawaii and Los Angeles was assessed in a calibration substudy that compared diet reported from the questionnaire with three 24-hour dietary recalls. For the calibration substudy, subjects from each of eight subgroups defined by sex and ethnic group (African-American, Japanese-American, Latino, and White) were chosen randomly from among the cohort members, and each participant's previous day's diet was assessed by telephone recall on three occasions over approximately 2 months. After completing the three 24-hour recalls, each calibration subject was sent a second questionnaire; 1,606 persons completed three recalls and a second questionnaire (127 to 267 per ethnic-sex group). This report describes correlation coefficients and calibration slopes for the relation between the 24-hour recalls and second questionnaire values for a selected set of macro- and micronutrients, as absolute intakes, nutrient densities, and calorie-adjusted nutrients. In all subgroups, estimates of the correlation between the questionnaire and 24-hour recalls were greater after energy adjustment (average correlations ranged from 0.57-0.74 for nutrient densities and from 0.55-0.74 for calorie-adjusted nutrients) than when absolute nutrient values were used (average range 0.26-0.57). For absolute nutrient intakes, the correlations were greatest for Whites, somewhat lower for Japanese-Americans and Latinos, and lowest for African-Americans. After energy adjustment, the difference between subgroups were diminished, and the correlations were generally highly satisfactory.

Measurement error in epidemiology: the design of validation studies II: bivariate situation

The bias in relative risk estimates caused by errors in measurement of the relevant exposure is being increasingly recognized in epidemiology. Estimation of the necessary correction factor to remove this bias for univariate exposure has been considered in an earlier paper. We consider here the multivariate situation in which non-differential errors in measurement can lead to incorrect identification of the variable most closely associated with disease. Estimation of the necessary correction factor when the true exposure is unobservable necessarily requires assumptions. We explore the robustness of the estimation to departures from a range of assumptions. The value of good biomarkers is demonstrated. We present a bivariate example in which failure to take account of measurement error leads to the incorrect exposure being identified as the important determinant of disease risk.

Measurement error in epidemiology: the design of validation studies I: univariate situation

It is becoming standard practice in epidemiology to adjust relative risk estimates to remove the bias caused by non-differential errors in the exposure measurement. Estimation of the correction factor is often based on a validation study incorporating repeated measures of exposure, which are assumed to be independent. This assumption is difficult to verify and often likely to be false. We examine the effect of departures from this assumption on the correction factor estimate, and explore the design of validation studies using two or even three different types of measurement of exposure, where assumption of independence between the measures may be more realistic. The value of good biomarker measures of exposure is demonstrated even if they are feasible to use only in a validation study.

Validation and comparison of two computerized methods of obtaining a diet history

The aim of this study was to validate two computerized methods of obtaining a diet history (DH and EBIS). The food consumption of 12 men and eight women was calculated by weighing each food item over a period of 8 days. Thereafter the diet history was taken over this period by using both programs alternatively. The intake of energy, protein, fat and carbohydrates, and 10 further nutrients was evaluated and the percentage difference calculated. In general, the intake of nutrients calculated from the diet history tended to be underestimated by most of the people interviewed. The mean daily intake of the nutrients calculated from the DH program deviates from -34% to +20% (mean SD = 48.1) and -35% to +15% for EBIS (mean SD = 28.1). In conclusion, both computerized methods proved useful for epidemiological studies, but not for the determination of deficiencies in individuals.

Reproducibility and relative validity of a self-administered semiquantitative food frequency questionnaire applied to younger women

We have evaluated the reproducibility and relative validity of a semiquantitative food frequency questionnaire (FFQ) used in a prospective study of risk factors for cervical neoplasia. The questionnaire is a modified version of one developed and evaluated in a middle-aged Danish population. In the present study, 122 women from the general population of Copenhagen, aged 20-29 years, completed the FFQ twice at a 1-year interval, and provided three 4-day dietary records during the intervening year. The mean nutrient intakes calculated from the first and second questionnaire were similar and, for most nutrients, close to those obtained from the dietary records. The Pearson correlation coefficients between the mean nutrient intakes from the two questionnaires ranged from 0.53 (95% CI, 0.39-0.65) for vitamin E to 0.76 (95% CI, 0.67-0.83) for vitamin B12 (median, 0.67 [95% CI, 0.56-0.76]). In comparisons between the second FFQ and the dietary records, the correlations ranged from 0.24 (95% CI, 0.07-0.40) for vitamin D to 0.63 (95% CI, 0.51-0.73) for sucrose (median, 0.42 [95% CI, 0.26-0.561). The correlations between FFQ and dietary records were generally higher after adjustment for energy intake (median, 0.53 [95% CI, 0.39-0.65]) and within-person variability (median, 0.64 [95% CI, 0.52-0.73]). On average, 71% of the women were classified in the same (+/- 1) quintile in the second FFQ and the dietary records. An average of 3.8% of the women were grossly misclassified into the highest and lowest quintiles by the dietary records. The relative validity of the FFQ in this population was similar to that reported earlier. It is concluded that the FFQ is reproducible and provides a useful scale for categorizing individuals according to their intake of energy and nutrients.

Validation of a food-frequency questionnaire to assess dietary intakes in cancer studies in Italy. Results for specific nutrients

The validity of a 77-item food-frequency questionnaire (FFQ) developed for a multicenter case-control study on diet and cancer in Italy was assessed. Trained interviewers administered the same FFQ to 452 volunteers from three Italian provinces (Pordenone, Genoa, and Forli) completed in two different seasons, at an interval of 3 to 10 months. For 395 (130 males, 265 females; median age = 52 years; range = 35 to 69 years) volunteers, two 7-day dietary (7-DD) records were available. Average intake obtained by means of the FFQ was overestimated by approximately 18% in comparison with the corresponding values based on the two 7-DD records (reference method). Pearson partial correlation coefficients, adjusted for total energy intake between the nutrient intakes assessed by the FFQ and reference method, ranged from 0.19 for vegetable fat to 0.64 for sugar (median value r = 0.46). The unadjusted deattenuated coefficients, which took into account the interindividual variability of consumption, estimated by means of the two 7-DD records, ranged from 0.29 for vegetable fat to 0.72 for starch (median value r = 0.54). The proportion of subjects correctly classified within the lowest two quintiles ranged between 59% for vegetable fat and vitamin E, and 96% for alcohol, and those correctly classified within the highest two quintiles ranged between 44% for vegetable fat and 94% for alcohol. The average proportion of subjects correctly classified within one quintile was 73%. These data indicate that this FFQ provides valid estimates of intakes for major nutrients, comparable to those reported from other studies in North America and other European countries.

Statistics in epidemiology: the case-control study

This article presents a general review of the major trends in the conceptualization, development, and success of case-control methods for the study of disease causation and prevention. "Recent work on nested case-control, case-cohort, and two-stage case control designs demonstrates the continuing impact of statistical thinking on epidemiology. The influence of R. A. Fisher's work on these developments is mentioned wherever possible. His objections to the drawing of causal conclusions from observational data on cigarette smoking and lung cancer are used to introduce the problems of measurement error and confounding bias."

Measurement error in dietary assessment: an investigation using covariance structure models. Part II

In Part I we presented a covariance structure model for analysing measurement error in the assessment of nitrogen intake. In this paper we include data on urine nitrogen excretion which allows a critical assessment of the model proposed. Inclusion of urine nitrogen data produces more pessimistic estimates of the quality of dietary intake measurements and shows that previous assumptions about independence of measurement error were wrong. This underscores the need for well founded assumptions in the analysis of measurement error.