Measurement error in dietary assessment: an investigation using covariance structure models. Part II

Validation of the Oxford WebQ Online 24-Hour Dietary Questionnaire Using Biomarkers

The Oxford WebQ is an online 24-hour dietary questionnaire that is appropriate for repeated administration in large-scale prospective studies, including the UK Biobank study and the Million Women Study. We compared the performance of the Oxford WebQ and a traditional interviewer-administered multiple-pass 24-hour dietary recall against biomarkers for protein, potassium, and total sugar intake and total energy expenditure estimated by accelerometry. We recruited 160 participants in London, United Kingdom, between 2014 and 2016 and measured their biomarker levels at 3 nonconsecutive time points. The measurement error model simultaneously compared all 3 methods. Attenuation factors for protein, potassium, total sugar, and total energy intakes estimated as the mean of 2 applications of the Oxford WebQ were 0.37, 0.42, 0.45, and 0.31, respectively, with performance improving incrementally for the mean of more measures. Correlation between the mean value from 2 Oxford WebQs and estimated true intakes, reflecting attenuation when intake is categorized or ranked, was 0.47, 0.39, 0.40, and 0.38, respectively, also improving with repeated administration. These correlations were similar to those of the more administratively burdensome interviewer-based recall. Using objective biomarkers as the standard, the Oxford WebQ performs well across key nutrients in comparison with more administratively burdensome interviewer-based 24-hour recalls. Attenuation improves when the average value is taken over repeated administrations, reducing measurement error bias in assessment of diet-disease associations.

Invited Commentary: The Contribution to the Field of Nutritional Epidemiology of the Landmark 1985 Publication by Willett et al

The semiquantitative food frequency questionnaire (FFQ) has been the primary source of dietary exposure data in epidemiology for decades. Although frequency instruments had been evaluated before the 1985 publication "Reproducibility and Validity of a Semiquantitative Food Frequency Questionnaire" by Willett et al. (Am J Epidemiol. 1985;122(1):51-65), that paper was the prototype for the development and validation of what was then a highly innovative method for collecting dietary data. This approach was adopted in nearly all subsequent cohort studies of diet and disease. The paper also catalyzed an extended scientific discourse regarding methods for validation, energy adjustment, and measurement error. It is now well established that data from FFQs and other self-reported dietary assessment instruments have both value and error and that this error should be considered in the analysis and interpretation of findings, including sensitivity analyses in which adjustment for measurement error is explored. Advances in technology make it feasible to consider collecting multiple granular short-term instruments such as recalls or records over time in addition to FFQs among all participants in large cohort studies; both provide valuable information. Without a doubt, the 1985 publication by Willett et al. provided the foundation that propelled the field of nutritional epidemiology forward, and it continues to be relevant today.

Correcting for covariate measurement error in logistic regression using nonparametric maximum likelihood estimation

When covariates are measured with error, inference based on conventional generalized linear models can yield biased estimates of regression parameters. This problem can potentially be rectified by using generalized linear latent and mixed models (GLLAMM), including a measurement model for the relationship between observed and true covariates. However, the models are typically estimated under the assumption that both the true covariates and the measurement errors are normally distributed, although skewed covariate distributions are often observed in practice. In this article we relax the normality assumption for the true covariates by developing nonparametric maximum likelihood estimation (NPMLE) for GLLAMMs. The methodology is applied to estimating the effect of dietary fibre intake on coronary heart disease. We also assess the performance of estimation of regression parameters and empirical Bayes prediction of the true covariate. Normal as well as skewed covariate distributions are simulated and inference is performed based on both maximum likelihood assuming normality and NPMLE. Both estimators are unbiased and have similar root mean square errors when the true covariate is normal. With a skewed covariate, the conventional estimator is biased but has a smaller mean square error than the NPMLE. NPMLE produces substantially improved empirical Bayes predictions of the true covariate when its distribution is skewed.

Methods to Assess Measurement Error in Questionnaires of Sedentary Behavior

Sedentary behavior has already been associated with mortality, cardiovascular disease, and cancer. Questionnaires are an affordable tool for measuring sedentary behavior in large epidemiological studies. Here, we introduce and evaluate two statistical methods for quantifying measurement error in questionnaires. Accurate estimates are needed for assessing questionnaire quality. The two methods would be applied to validation studies that measure a sedentary behavior by both questionnaire and accelerometer on multiple days. The first method fits a reduced model by assuming the accelerometer is without error, while the second method fits a more complete model that allows both measures to have error. Because accelerometers tend to be highly accurate, we show that ignoring the accelerometer's measurement error, can result in more accurate estimates of measurement error in some scenarios. In this manuscript, we derive asymptotic approximations for the Mean-Squared Error of the estimated parameters from both methods, evaluate their dependence on study design and behavior characteristics, and offer an R package so investigators can make an informed choice between the two methods. We demonstrate the difference between the two methods in a recent validation study comparing Previous Day Recalls (PDR) to an accelerometer-based ActivPal.

Addressing Current Criticism Regarding the Value of Self-Report Dietary Data

Recent reports have asserted that, because of energy underreporting, dietary self-report data suffer from measurement error so great that findings that rely on them are of no value. This commentary considers the amassed evidence that shows that self-report dietary intake data can successfully be used to inform dietary guidance and public health policy. Topics discussed include what is known and what can be done about the measurement error inherent in data collected by using self-report dietary assessment instruments and the extent and magnitude of underreporting energy compared with other nutrients and food groups. Also discussed is the overall impact of energy underreporting on dietary surveillance and nutritional epidemiology. In conclusion, 7 specific recommendations for collecting, analyzing, and interpreting self-report dietary data are provided: (1) continue to collect self-report dietary intake data because they contain valuable, rich, and critical information about foods and beverages consumed by populations that can be used to inform nutrition policy and assess diet-disease associations; (2) do not use self-reported energy intake as a measure of true energy intake; (3) do use self-reported energy intake for energy adjustment of other self-reported dietary constituents to improve risk estimation in studies of diet-health associations; (4) acknowledge the limitations of self-report dietary data and analyze and interpret them appropriately; (5) design studies and conduct analyses that allow adjustment for measurement error; (6) design new epidemiologic studies to collect dietary data from both short-term (recalls or food records) and long-term (food-frequency questionnaires) instruments on the entire study population to allow for maximizing the strengths of each instrument; and (7) continue to develop, evaluate, and further expand methods of dietary assessment, including dietary biomarkers and methods using new technologies.

Considering the value of dietary assessment data in informing nutrition-related health policy

Dietary assessment has long been known to be challenged by measurement error. A substantial amount of literature on methods for determining the effects of error on causal inference has accumulated over the past decades. These methods have unrealized potential for improving the validity of data collected for research studies and national nutritional surveillance, primarily through the NHANES. Recently, the validity of dietary data has been called into question. Arguments against using dietary data to assess diet-health relations or to inform the nutrition policy debate are subject to flaws that fall into 2 broad areas: 1) ignorance or misunderstanding of methodologic issues; and 2) faulty logic in drawing inferences. Nine specific issues are identified in these arguments, indicating insufficient grasp of the methods used for assessing diet and designing nutritional epidemiologic studies. These include a narrow operationalization of validity, failure to properly account for sources of error, and large, unsubstantiated jumps to policy implications. Recent attacks on the inadequacy of 24-h recall-derived data from the NHANES are uninformative regarding effects on estimating risk of health outcomes and on inferences to inform the diet-related health policy debate. Despite errors, for many purposes and in many contexts, these dietary data have proven to be useful in addressing important research and policy questions. Similarly, structured instruments, such as the food frequency questionnaire, which is the mainstay of epidemiologic literature, can provide useful data when errors are measured and considered in analyses.

Lifetime body size and reproductive factors: comparisons of data recorded prospectively with self reports in middle age

Background

Data on lifetime exposures are often self-reported in epidemiologic studies, sometimes many years after the relevant age. Validity of self-reported data is usually inferred from their agreement with measured values, but few studies directly quantify the likely effects of reporting errors in body size and reproductive history variables on estimates of disease-exposure associations.

Methods

The MRC National Survey of Health and Development (NSHD) and the Million Women Study (MWS) are UK population-based prospective cohorts. The NSHD recruited participants at birth in 1946 and has followed them at regular intervals since then, whereas the MWS recruited women in middle age. For 541 women who were participants in both studies, we used statistical measures of association and agreement to compare self-reported MWS data on body size throughout life and reproductive history, obtained in middle age, to NSHD data measured or reported close to the relevant ages. Likely attenuation of estimates of linear disease-exposure associations due to the combined effects of random and systematic errors was quantified using regression dilution ratios (RDRs).

Results

Data from the two studies were very strongly correlated for current height, weight and body mass index, and age at menopause (Pearson r = 0.91-0.95), strongly correlated for birth weight, parental heights, current waist and hip circumferences and waist-to-height ratio (r = 0.67-0.80), and moderately correlated for age at menarche and waist-to-hip ratio (r = 0.52-0.57). Self-reported categorical body size and clothes size data for various ages were moderately to strongly associated with anthropometry collected at the relevant times (Spearman correlations 0.51-0.79). Overall agreement between the studies was also good for most quantitative variables, although all exhibited both random and systematic reporting error. RDRs ranged from 0.66 to 0.86 for most variables (slight to moderate attenuation), except weight and body mass index (1.02 and 1.04, respectively; little or no attenuation), and age at menarche, birth weight and waist-to-hip ratio (0.44, 0.59 and 0.50, respectively; substantial attenuation).

Conclusions

This study provides some evidence that self-reported data on certain anthropometric and reproductive factors may be adequate for describing disease-exposure associations in large epidemiological studies, provided that the effects of reporting errors are quantified and the results are interpreted with caution.

Measurement error correction for nutritional exposures with correlated measurement error: use of the method of triads in a longitudinal setting

Nutritional exposures are often measured with considerable error in commonly used surrogate instruments such as the food frequency questionnaire (FFQ) (denoted by Q(i) for the ith subject). The error can be both systematic and random. The diet record (DR) denoted by R(i) for the ith subject is considered an alloyed gold standard. However, some authors have reported both systematic and random errors with this instrument as well.One goal in measurement error research is to estimate the regression coefficient of T(i) (true intake for the ith subject) on Q(i) denoted by lambda(TQ). If the systematic errors in Q(i) and R(i) (denoted by q(i) and r(i)) are uncorrelated, then one can obtain an unbiased estimate of lambda(TQ) by lambda(RQ) obtained by regressing R(i) on Q(i). However, if Corr(q(i), r(i))>0, then lambda(RQ)>lambda(TQ).In this paper, we propose a method for indirectly estimating lambda(TQ) even in the presence of correlated systematic error based on a longitudinal design where Q(i) (surrogate measure of dietary intake), R(i) (a reference measure of dietary intake), and M(i) (a biomarker) are available on the same subjects at 2 time points. In addition, between-person variation in mean levels of M(i) among people with the same dietary intake is also accounted for. The methodology is illustrated for dietary vitamin C intake based on longitudinal data from 323 subjects in the European Prospective Investigation of Cancer (EPIC)-Norfolk study who provided two measures of dietary vitamin C intake from the FFQ (Q(i)) and a 7-day DR (R(i)) and plasma vitamin C (M(i)) 4 years apart.

Fat, protein, and meat consumption and renal cell cancer risk: a pooled analysis of 13 prospective studies

BACKGROUND

Results of several case-control studies suggest that high consumption of meat (all meat, red meat, or processed meat) is associated with an increased risk of renal cell cancer, but only a few prospective studies have examined the associations of intakes of meat, fat, and protein with renal cell cancer.

METHODS

We conducted a pooled analysis of 13 prospective studies that included 530,469 women and 244,483 men and had follow-up times of up to 7-20 years to examine associations between meat, fat, and protein intakes and the risk of renal cell cancer. All participants had completed a validated food frequency questionnaire at study entry. Using the primary data from each study, we calculated the study-specific relative risks (RRs) for renal cell cancer by using Cox proportional hazards models and then pooled these RRs by using a random-effects model. All statistical tests were two-sided.

RESULTS

A total of 1,478 incident cases of renal cell cancer were identified (709 in women and 769 in men). We observed statistically significant positive associations or trends in pooled age-adjusted models for intakes of total fat, saturated fat, monounsaturated fat, polyunsaturated fat, cholesterol, total protein, and animal protein. However, these associations were attenuated and no longer statistically significant after adjusting for body mass index, fruit and vegetable intake, and alcohol intake. For example, the pooled age-adjusted RR of renal cell cancer for the highest vs the lowest quintile of intake for total fat was 1.30 (95% confidence interval [CI] = 1.08 to 1.56; P(trend) = .001) and for total protein was 1.17 (95% CI = 0.99 to 1.38; P(trend) = .02). By comparison, the pooled multivariable RR for the highest vs the lowest quintile of total fat intake was 1.10 (95% CI = 0.92 to 1.32; P(trend) = .31) and of total protein intake was 1.06 (95% CI = 0.89 to 1.26; P(trend) = .37). Intakes of red meat, processed meat, poultry, or seafood were not associated with the risk of renal cell cancer.

CONCLUSIONS

Intakes of fat and protein or their subtypes, red meat, processed meat, poultry, and seafood are not associated with risk of renal cell cancer.

Classical latent variable models for medical research

Latent variable models are commonly used in medical statistics, although often not referred to under this name. In this paper we describe classical latent variable models such as factor analysis, item response theory, latent class models and structural equation models. Their usefulness in medical research is demonstrated using real data. Examples include measurement of forced expiratory flow, measurement of physical disability, diagnosis of myocardial infarction and modelling the determinants of clients' satisfaction with counsellors' interviews.

Performance of a food-frequency questionnaire in the US NIH-AARP (National Institutes of Health-American Association of Retired Persons) Diet and Health Study

OBJECTIVE

We evaluated the performance of the food-frequency questionnaire (FFQ) administered to participants in the US NIH-AARP (National Institutes of Health-American Association of Retired Persons) Diet and Health Study, a cohort of 566 404 persons living in the USA and aged 50-71 years at baseline in 1995.

DESIGN

The 124-item FFQ was evaluated within a measurement error model using two non-consecutive 24-hour dietary recalls (24HRs) as the reference.

SETTING

Participants were from six states (California, Florida, Pennsylvania, New Jersey, North Carolina and Louisiana) and two metropolitan areas (Atlanta, Georgia and Detroit, Michigan).

SUBJECTS

A subgroup of the cohort consisting of 2053 individuals.

RESULTS

For the 26 nutrient constituents examined, estimated correlations with true intake (not energy-adjusted) ranged from 0.22 to 0.67, and attenuation factors ranged from 0.15 to 0.49. When adjusted for reported energy intake, performance improved; estimated correlations with true intake ranged from 0.36 to 0.76, and attenuation factors ranged from 0.24 to 0.68. These results compare favourably with those from other large prospective studies. However, previous biomarker-based studies suggest that, due to correlation of errors in FFQs and self-report reference instruments such as the 24HR, the correlations and attenuation factors observed in most calibration studies, including ours, tend to overestimate FFQ performance.

CONCLUSION

The performance of the FFQ in the NIH-AARP Diet and Health Study, in conjunction with the study's large sample size and wide range of dietary intake, is likely to allow detection of moderate (> or =1.8) relative risks between many energy-adjusted nutrients and common cancers.

Both food habit change in the past and obesity status may influence the association between dietary factors and postmenopausal breast cancer

OBJECTIVE

Valid dietary data are essential when trying to identify whether or not one or more dietary exposures are responsible for disease. We examined diet composition in women who reported dietary change in the past compared with non-changers, and how the associations between dietary factors and postmenopausal breast cancer are influenced by dietary change, obesity status and misreporting of energy.

DESIGN

A population-based prospective cohort study. Data were obtained by a diet history method, anthropometrical measurements and an extensive lifestyle questionnaire including items on past food habit change.

SETTING

The Malmö Diet and Cancer (MDC) study, conducted in Malmö, Sweden.

SUBJECTS

A subsample of 12,781 women from the MDC cohort recruited from 1991 to 1996. A total of 428 postmenopausal women were diagnosed with incident breast cancer, during 9.2 years of follow-up.

RESULTS

Past food habit changers reported healthier food habits and lower energy intake compared with non-changers, a finding that raises issues regarding possible reporting biases. When excluding diet changers, the trend of increased breast cancer risk across omega-6 fatty acid quintiles was stronger, and a tendency of decreased risk emerged for 'fruit, berries and vegetables'. When excluding individuals with non-adequate reports of energy intake, risk estimates were similar to that of the whole sample. In women with body mass index < 27 kg m- 2, significant trends of increased breast cancer risk were seen for total fat and omega-6 fatty acids, and of decreased risk for 'fruit, berries and vegetables'.

CONCLUSIONS

This study indicates that both obesity and self-reported past food habit change may be important confounders of diet-breast cancer relationships. The study demonstrates that sensitivity analysis, through stratification, may facilitate interpretation of risk relationships and study results.

Bayesian statistics in medicine: a 25 year review

This review examines the state of Bayesian thinking as Statistics in Medicine was launched in 1982, reflecting particularly on its applicability and uses in medical research. It then looks at each subsequent five-year epoch, with a focus on papers appearing in Statistics in Medicine, putting these in the context of major developments in Bayesian thinking and computation with reference to important books, landmark meetings and seminal papers. It charts the growth of Bayesian statistics as it is applied to medicine and makes predictions for the future. From sparse beginnings, where Bayesian statistics was barely mentioned, Bayesian statistics has now permeated all the major areas of medical statistics, including clinical trials, epidemiology, meta-analyses and evidence synthesis, spatial modelling, longitudinal modelling, survival modelling, molecular genetics and decision-making in respect of new technologies.

Correlations between estimated and true dietary intakes: using two instrumental variables

PURPOSE

We describe a new application of the method of triads that allows an estimate of the correlation between a dietary questionnaire measure (Q) and true intake (T).

METHODS

Three surrogate variables Q, M, and P are observed where M and P are both instrumental (often biological) variables. A reference dietary method (R) is not required. The variables M and P may be concentration rather than recovery biomarkers. Estimating equations produce Corr(Q,T), Corr(M,T), Corr(P,T), conditional on assumptions about error correlations. Correlations between errors in both Q and a reference dietary measure can also be estimated if R is available. A small validation study of California Seventh-day Adventists provided food frequency, repeated 24-hour dietary recalls (R), and biological data (blood, overnight urines, and subcutaneous fat).

RESULTS

Values of Corr(Q,T) ranged between 0.40 and 0.66. Values of Corr(R,T) were higher, between 0.48 and 0.83. Estimated correlations between errors in R and Q were all positive.

CONCLUSIONS

When carefully chosen, M and P, rather than M and R, should better satisfy assumptions about error correlations. Food frequency data and repeated 24-hour recalls both provide estimates of T, but the latter has greater validity. Standard errors suggest that for good precision Corr(Q,T) requires large validation studies (2000-3000 subjects).

Fiber intake and incidence of colorectal cancer among 76,947 women and 47,279 men

Prospective cohort studies have consistently found no important link between fiber intake and risk of colorectal cancer. The recent large, prospective European Prospective Investigation into Cancer and Nutrition has challenged this paradigm by suggesting significant protection by high fiber intake. We prospectively investigated the association of fiber intake with the incidence of colon and rectal cancers in two large cohorts: the Nurses' Health Study (76,947 women) and the Health Professionals Follow-up Study (47,279 men). Diet was assessed repeatedly in 1984, 1986, 1990, and 1994 among women and in 1986, 1990, and 1994 among men. The incidence of cancer of the colon and rectum was ascertained up to the year 2000. Relative risk estimates were calculated using a Cox proportional hazards model simultaneously controlling for potential confounding variables. During follow-up including 1.8 million person-years and 1,596 cases of colorectal cancer, we found little association with fiber intake after controlling for confounding variables. The hazard ratio for a 5-g/d increase in fiber intake was 0.91 (95% confidence interval, 0.87-0.95) after adjusting for covariates used in the European Prospective Investigation into Cancer and Nutrition study and 0.99 (95% confidence interval, 0.95-1.04) after adjusting for additional confounding variables. Our data from two large prospective cohorts with long follow-up and repeated assessment of fiber intake and of a large number of potential confounding variables do not indicate an important association between fiber intake and colorectal cancer but reveal considerable confounding by other dietary and lifestyle factors.

Correlations between estimated and true dietary intakes

PURPOSE

It is unclear how well questionnaire or so-called reference methods of dietary assessment correlate with true dietary intake. We develop a method to estimate such correlations.

METHODS

An error model is described that uses data from a food frequency questionnaire (Q), a reference method (R), and a biological marker (M). The model does not assume the classical error model for either R or M, or that the correlation between errors in the questionnaire and reference data is zero. Credible intervals can be placed about correlations between R, Q, M and true dietary data (T), also about the correlations between errors in reference and questionnaire data.

RESULTS

Application of this model to a validation data set mainly found correlations in the range 0.4 to 0.8, and that correlations (R,T) generally exceeded correlations (Q,T), providing evidence that R is more valid than Q. Estimated correlations between errors in R and Q were often far from zero suggesting that regression calibration to imperfect reference data is problematic unless these error correlations can be estimated.

CONCLUSION

A biological marker in addition to dietary data, allows calculation of correlations between estimated and true dietary intakes under reasonable assumptions about errors. However, sensitivity analyses are necessary on one variable.

Markers of the validity of reported energy intake

Energy intake (EI) is the foundation of the diet, because all other nutrients must be provided within the quantity of food needed to fulfill the energy requirement. Thus if total EI is underestimated, it is probable that the intakes of other nutrients are also underestimated. Under conditions of weight stability, EI equals energy expenditure (EE). Because at the group level weight may be regarded as stable in the timescale of a dietary assessment, the validity of reported EI can be evaluated by comparing it with either measured EE or an estimate of the energy requirement of the population. This paper provides the first comprehensive review of studies in which EI was reported and EE was measured using the doubly labeled water technique. These conclusively demonstrate widespread bias to the underestimation of EI. Because energy requirements of populations or individuals can be conveniently expressed as multiples of the basal metabolic rate (BMR), EE:BMR, reported EI may also be expressed as EI:BMR for comparison. Values of EI:BMR falling below the 95% confidence limit of agreement between these two measures signify the presence of underreporting. A formula for calculating the lower 95% confidence limit was proposed by Goldberg et al. (the Goldberg cutoff). It has been used by numerous authors to identify individual underreporters in different dietary databases to explore the variables associated with underreporting. These studies are also comprehensively reviewed. They explore the characteristics of underreporters and the biases in estimating nutrient intake and in describing meal patterns associated with underreporting. This review also examines some of the problems for the interpretation of data introduced by underreporting and particularly by variable underreporting across subjects. Future directions for research are identified.

Uses and limitations of statistical accounting for random error correlations, in the validation of dietary questionnaire assessments

OBJECTIVE

To examine statistical models that account for correlation between random errors of different dietary assessment methods, in dietary validation studies.

SETTING

In nutritional epidemiology, sub-studies on the accuracy of the dietary questionnaire measurements are used to correct for biases in relative risk estimates induced by dietary assessment errors. Generally, such validation studies are based on the comparison of questionnaire measurements (Q) with food consumption records or 24-hour diet recalls (R). In recent years, the statistical analysis of such studies has been formalized more in terms of statistical models. This made the need of crucial model assumptions more explicit. One key assumption is that random errors must be uncorrelated between measurements Q and R, as well as between replicate measurements R1 and R2 within the same individual. These assumptions may not hold in practice, however. Therefore, more complex statistical models have been proposed to validate measurements Q by simultaneous comparisons with measurements R plus a biomarker M, accounting for correlations between the random errors of Q and R.

CONCLUSIONS

The more complex models accounting for random error correlations may work only for validation studies that include markers of diet based on physiological knowledge about the quantitative recovery, e.g. in urine, of specific elements such as nitrogen or potassium, or stable isotopes administered to the study subjects (e.g. the doubly labelled water method for assessment of energy expenditure). This type of marker, however, eliminates the problem of correlation of random errors between Q and R by simply taking the place of R, thus rendering complex statistical models unnecessary.

Food frequency questionnaire and a screening test

We assessed the accuracy of a 141-item food frequency questionnaire as a screening test to detect high or low consumption of nutrients associated with cancer. Fifty-five men and 58 women participating in two population-based cohort studies in Miyagi, Japan, provided four three-day diet records over a one-year period and subsequently completed the questionnaire twice with a one-year interval. Pearson correlation coefficients between 17 nutrients measured by the diet records and the first questionnaire ranged from 0.24 to 0.85 (median 0.43), and those between the two questionnaires ranged from 0.47 to 0.91 (median 0.68). The sensitivity and specificity of the questionnaire for detecting high-alcohol, high-fat, low-calcium, and low-ascorbic acid consumers were 86.7% and 96.7%, 50.0% and 85.7%, 48.8% and 76.4%, and 61.9% and 70.0%, respectively. Receiver operating characteristic curves indicated comparable performance of the questionnaire and a three-day diet record, regarded as another screening test. The questionnaire performed poorly for other nutrients. The results indicate that our questionnaire is reasonably reproducible, comparable with the diet records, and useful as a screening test to detect high or low consumers of several nutrients associated with cancer for subsequent enrollment in dietary intervention trials or dietary counseling.

Biased over- or under-reporting is characteristic of individuals whether over time or by different assessment methods

Seven studies with repeated measurements of energy intake and/or nitrogen intake were examined to determine whether misreporting is characteristic of some persons or occurs randomly. Four of the studies were validated by doubly labeled water measurements of energy expenditure. Reporting validity was expressed as the ratio of energy intake to energy expenditure. Ratios were consistently below the expected value of 1.0 for some subjects and consistently above 1.0 for others, indicating characteristic reporting validity within subjects. Two year-long studies provided 4 to 12 measurements and a total number of days sufficient to measure individual habitual intake. Subjects mean energy intake to basal metabolic rate (BMR) ratios were < 1.35 in 45% and 47% and < 1.35 at every measurement in 25% of subjects. This indicated persistent underreporting over time, because 1.35 x BMR is the minimum energy expenditure compatible with a normally active lifestyle. Three of the studies used more than 1 assessment method (validated by doubly labeled water and/or urinary nitrogen excretion). There was a tendency for persons determined to be underreporters by 1 method to be also underreporters when tested by other methods. We conclude that biased over- or underreporting is characteristic of some persons. Thus, repeat measurements do not necessarily provide valid measures of individual intake, extreme intakes may reflect under- and overreporting rather than true low or high intakes, and subjects most prone to reporting bias may be repeatedly misclassified in quantiles of the distribution. This presents a challenge to dietitians nutritionists, and statisticians both for the design of surveys and the handling of flawed data.

Measurement error in epidemiology: the design of validation studies II: bivariate situation

The bias in relative risk estimates caused by errors in measurement of the relevant exposure is being increasingly recognized in epidemiology. Estimation of the necessary correction factor to remove this bias for univariate exposure has been considered in an earlier paper. We consider here the multivariate situation in which non-differential errors in measurement can lead to incorrect identification of the variable most closely associated with disease. Estimation of the necessary correction factor when the true exposure is unobservable necessarily requires assumptions. We explore the robustness of the estimation to departures from a range of assumptions. The value of good biomarkers is demonstrated. We present a bivariate example in which failure to take account of measurement error leads to the incorrect exposure being identified as the important determinant of disease risk.

Measurement error in epidemiology: the design of validation studies I: univariate situation

It is becoming standard practice in epidemiology to adjust relative risk estimates to remove the bias caused by non-differential errors in the exposure measurement. Estimation of the correction factor is often based on a validation study incorporating repeated measures of exposure, which are assumed to be independent. This assumption is difficult to verify and often likely to be false. We examine the effect of departures from this assumption on the correction factor estimate, and explore the design of validation studies using two or even three different types of measurement of exposure, where assumption of independence between the measures may be more realistic. The value of good biomarker measures of exposure is demonstrated even if they are feasible to use only in a validation study.

Evaluating epidemiologic evidence of the effects of food and nutrient exposures

The objective of this paper is to discuss some of the issues to be considered when evaluating and interpreting epidemiologic evidence from observational studies that collect data on dietary intake. The assessment of such evidence should include consideration of the study design, sample selection, and the measurements of exposure and disease. The degree and type of error in nutrient data can lead to analytic problems and potentially be a source of bias either toward or away from the null value. Because methods of statistical correction and adjustment for error, such as energy adjustment, cannot necessarily completely compensate for sources of bias in dietary data, additional research should be conducted on sources of error in dietary data. Published research using reported dietary data should include a discussion of potential sources of error and their effect on the results. The most useful studies are likely to be those designed to address a clearly defined prior hypothesis about a specific diet-disease relation. Because of the potential for bias and confounding, observational epidemiologic studies of diet and outcome cannot generally provide decisive evidence by themselves either for or against specific hypotheses. Although randomized clinical trials of the effects of specific nutrients or dietary modifications are not always feasible, they provide more definitive results and should generally be considered more valid than observational studies using self-reported dietary intake. Well-designed observational epidemiologic studies using self-reported dietary intake can provide valuable data to support or challenge hypotheses derived from clinical or laboratory data and to suggest further directions for investigation.

Validation and comparison of two computerized methods of obtaining a diet history

The aim of this study was to validate two computerized methods of obtaining a diet history (DH and EBIS). The food consumption of 12 men and eight women was calculated by weighing each food item over a period of 8 days. Thereafter the diet history was taken over this period by using both programs alternatively. The intake of energy, protein, fat and carbohydrates, and 10 further nutrients was evaluated and the percentage difference calculated. In general, the intake of nutrients calculated from the diet history tended to be underestimated by most of the people interviewed. The mean daily intake of the nutrients calculated from the DH program deviates from -34% to +20% (mean SD = 48.1) and -35% to +15% for EBIS (mean SD = 28.1). In conclusion, both computerized methods proved useful for epidemiological studies, but not for the determination of deficiencies in individuals.

Energy adjustment does not control for differential recall bias in nutritional epidemiology

It has been stated that energy adjustment can control for recall bias in case-control studies. Simulation of recall bias and cases and controls in a nutritional survey of German adults was conducted to examine its impact on five dietary effects, (adding a macronutrient, substituting one macronutrient for another, adding a macronutrient while keeping the other energy sources constant, and changing the macronutrient to energy ratio through addition or substitution) using various energy adjustment models. If energy adjustment were an effective means of correcting measurement error, the energy adjusted dietary effects, after a subtraction of energy and fat intake, should equal those in the original data set. Simulation of differential under-reporting of fat and energy intake by cases but not controls showed this to dramatically impact all five considered dietary effects, even after energy adjustment. The influence of the assumed recall bias on the different effects depends on the error type structure, inflating an odds ration of 1.8 to as much as 12.3 or reducing it to 0.45 when 100 kcal of fat was substituted for 100 kcal of other macronutrients. Although energy adjustment may serve many functions, it cannot correct for differential error. Depending upon the nature of the hypothesized effect and the error type, energy adjustment may also distort risk ratios in the presence of non-differential bias. The concern that cases and controls report their energy intakes with different degrees of error remains a critical consideration that must be addressed through improved measurements, and not energy adjustment under any of the currently used models.

Validation of a food-frequency questionnaire to assess dietary intakes in cancer studies in Italy. Results for specific nutrients

The validity of a 77-item food-frequency questionnaire (FFQ) developed for a multicenter case-control study on diet and cancer in Italy was assessed. Trained interviewers administered the same FFQ to 452 volunteers from three Italian provinces (Pordenone, Genoa, and Forli) completed in two different seasons, at an interval of 3 to 10 months. For 395 (130 males, 265 females; median age = 52 years; range = 35 to 69 years) volunteers, two 7-day dietary (7-DD) records were available. Average intake obtained by means of the FFQ was overestimated by approximately 18% in comparison with the corresponding values based on the two 7-DD records (reference method). Pearson partial correlation coefficients, adjusted for total energy intake between the nutrient intakes assessed by the FFQ and reference method, ranged from 0.19 for vegetable fat to 0.64 for sugar (median value r = 0.46). The unadjusted deattenuated coefficients, which took into account the interindividual variability of consumption, estimated by means of the two 7-DD records, ranged from 0.29 for vegetable fat to 0.72 for starch (median value r = 0.54). The proportion of subjects correctly classified within the lowest two quintiles ranged between 59% for vegetable fat and vitamin E, and 96% for alcohol, and those correctly classified within the highest two quintiles ranged between 44% for vegetable fat and 94% for alcohol. The average proportion of subjects correctly classified within one quintile was 73%. These data indicate that this FFQ provides valid estimates of intakes for major nutrients, comparable to those reported from other studies in North America and other European countries.

Measurement error in dietary assessment: an investigation using covariance structure models. Part I

Repeated measures of diet are analysed in an attempt to discover the measurement error properties of various dietary assessment methods. It is customary in such studies to assume that one reference measurement is 'valid' (without error) but this assumption is not tenable. An alternative approach, widely used in social science, is to model the covariance matrix of the repeated measures. We apply this methodology to assessments of nitrogen intake, which is essentially equivalent to protein intake.