Comparison of two expert-based assessments of diesel exhaust exposure in a case-control study: programmable decision rules versus expert review of individual jobs

Psychosocial work characteristic profiles and health outcomes in registered nurses at different stages of their careers: a cross-sectional study

BACKGROUND

Individual psychosocial work characteristics have been associated with the health and well-being of registered nurses. However, it remains to be determined whether different types of psychosocial work characteristics form patterned profiles and whether the profiles are associated with registered nurses' health and welfare at different stages of their careers. The purpose of this study was to identify latent psychosocial work characteristic profiles and examine whether the profiles are associated with a certain career stage and health outcomes.

METHODS

This cross-sectional study was conducted with 624 early-career registered nurses and 1,016 later-career registered nurses. Data were collected using an electronic survey with internationally validated measures including the Organizational Justice Scale, the Nurse Stress Index Scale, the Job Content Questionnaire, the Team Climate Inventory, the Psychological Distress Questionnaire, the Sleep Problems Questionnaire, and the Self-Rated Health Questionnaire. Latent profile analysis was conducted to identify subgroups with similar psychosocial work characteristic profiles. Multinomial and linear regression analyses were used to examine the association between latent work characteristics profiles, stage of career, and health outcomes.

RESULTS

We identified five profiles. The profiles were named based on class descriptions. The low strain/high support profile group and the moderate strain/high support profile group had statistically better self-rated health (p = < 0.001), less psychological distress (p = < 0.001) and less sleep problems (p = < 0.001) compared to the high strain/low support profile group.

CONCLUSIONS

Low to moderate strain, high interactional and procedural justice, and participative safety in teams form patterned profiles associated with better health in registered nurses. High strain, a lack of justice and a lack of participation safety form a risk combination pattern profile that may lead to health problems in registered nurses. Promoting procedural and interactional justice, and participation safety in teams seems efficient in enhancing the health and well-being of registered nurses. The findings indicate no significant correlation between career stages and work characteristic profiles. It is crucial to identify stressors specific for career stages and develop tailored interventions.

Comparison of agreement in asthmagen exposure assessments between rule-based automatic algorithms and a job exposure matrix in healthcare workers in Australia and Bhutan

Background:

Assessment of occupational exposures is an integral component of population-based studies investigating the epidemiology of occupational diseases. However, all the available methods for exposure assessment have been developed, tested and used in high-income countries. Except for a few studies examining pesticide exposures, there is limited research on whether these methods are appropriate for assessing exposure in LMICs. The aim of this study is to compare a task-specific algorithm-based method (OccIDEAS) to a job-specific matrix method (OAsJEM) in the assessment of asthmagen exposures among healthcare workers in a high-income country and a low- and middle- income country (LMIC) to determine an appropriate assessment method for use in LMICs for future research.

Methods:

Data were obtained from a national cross-sectional survey of occupational asthmagens exposure in Australia and a cross-sectional survey of occupational chemical exposure among Bhutanese healthcare workers. Exposure was assessed using OccIDEAS and the OAsJEM. Prevalence of exposure to asthmagens and inter-rater agreement were calculated.

Results:

In Australia, the prevalence was higher for a majority of agents when assessed by OccIDEAS than by the OAsJEM (13 versus 3). OccIDEAS identified exposures to a greater number of agents (16 versus 7). The agreement as indicated by κ (Cohen’s Kappa coefficient) for six of the seven agents assessed was poor to fair (0.02 to 0.37). In Bhutan, the prevalence of exposure assessed by OccIDEAS was higher for four of the seven agents and κ was poor for all the four agents assessed (-0.06 to 0.13). The OAsJEM overestimated exposures to high-level disinfectants by assigning exposures to all participants from 10 (Bhutan) and 12 (Australia) ISCO-88 codes; whereas OccIDEAS assigned exposures to varying proportions of participants from these ISCO-codes.

Conclusion:

There was poor to fair agreement in the assessment of asthmagen exposure in healthcare workers between the two methods. The OAsJEM overestimated the prevalence of certain exposures. As compared to the OAsJEM, OccIDEAS appeared to be more appropriate for evaluating cross-country exposures to asthmagens in healthcare workers due to its inherent quality of assessing task-based determinants and its versatility in being adaptable for use in different countries with different exposure circumstances.

Adapting Decision Rules to Estimate Occupational Metalworking Fluid Exposure in a Case-Control Study of Bladder Cancer in Spain

OBJECTIVES

We adapted previously developed decision rules from the New England Bladder Cancer Study (NEBCS) to assign occupational exposure to straight, soluble, and synthetic metalworking fluids (MWFs) to participants of the Spanish Bladder Cancer Study (SBCS).

METHODS

The SBCS and NEBCS are case-control studies that used the same lifetime occupational history and job module questionnaires. We adapted published decision rules from the NEBCS that linked questionnaire responses to estimates of the probability (<5, ≥5 to <50, ≥50 to <100, and 100%), frequency (in h week-1), and intensity (in mg m-3) of exposure to each of the three broad classes of MWFs to assign exposure to 10 182 reported jobs in the SBCS. The decision rules used the participant's module responses to MWF questions wherever possible. We then used these SBCS module responses to calculate job-, industry-, and time-specific patterns in the prevalence and frequency of MWF exposure. These estimates replaced the NEBCS-specific estimates in decision rules applied to jobs without MWF module responses. Intensity estimates were predicted using a previously developed statistical model that used the decade, industry (three categories), operation (grinding versus machining), and MWF type extracted from the SBCS questionnaire responses. We also developed new decision rules to assess mineral oil exposure from non-machining sources (possibly exposed versus not exposed). The decision rules for MWF and mineral oil identified questionnaire response patterns that required job-by-job expert review.

RESULTS

To assign MWF exposure, we applied decision rules that incorporated participant's responses and job group patterns for 99% of the jobs and conducted expert review of the remaining 1% (145) jobs. Overall, 14% of the jobs were assessed as having ≥5% probability of exposure to at least one of the three MWFs. Probability of exposure of ≥50% to soluble, straight, and synthetic MWFs was identified in 2.5, 1.7, and 0.5% of the jobs, respectively. To assign mineral oil from non-machining sources, we used module responses for 49% of jobs, a job-exposure matrix for 41% of jobs, and expert review for the remaining 10%. We identified 24% of jobs as possibly exposed to mineral oil from non-machining sources.

CONCLUSIONS

We demonstrated that we could adapt existing decision rules to assess exposure in a new population by deriving population-specific job group patterns.

Testing and Validating Semi-automated Approaches to the Occupational Exposure Assessment of Polycyclic Aromatic Hydrocarbons

INTRODUCTION

When it is not possible to capture direct measures of occupational exposure or conduct biomonitoring, retrospective exposure assessment methods are often used. Among the common retrospective assessment methods, assigning exposure estimates by multiple expert rater review of detailed job descriptions is typically the most valid, but also the most time-consuming and expensive. Development of screening protocols to prioritize a subset of jobs for expert rater review can reduce the exposure assessment cost and time requirement, but there is often little data with which to evaluate different screening approaches. We used existing job-by-job exposure assessment data (assigned by consensus between multiple expert raters) from a large, population-based study of women to create and test screening algorithms for polycyclic aromatic hydrocarbons (PAHs) that would be suitable for use in other population-based studies.

METHODS

We evaluated three approaches to creating a screening algorithm: a machine-learning algorithm, a set of a priori decision rules created by experts based on features (such as keywords) found in the job description, and a hybrid algorithm incorporating both sets of criteria. All coded jobs held by mothers of infants participating in National Birth Defects Prevention Study (NBDPS) (n = 35,424) were used in developing or testing the screening algorithms. The job narrative fields considered for all approaches included job title, type of product made by the company, main activities or duties, and chemicals or substances handled. Each screening approach was evaluated against the consensus rating of two or more expert raters.

RESULTS

The machine-learning algorithm considered over 30,000 keywords and industry/occupation codes (separate and in combination). Overall, the hybrid method had a similar sensitivity (87.1%) as the expert decision rules (85.5%) but was higher than the machine-learning algorithm (67.7%). Specificity was best in the machine-learning algorithm (98.1%), compared to the expert decision rules (89.2%) and hybrid approach (89.1%). Using different probability cutoffs in the hybrid approach resulted in improvements in sensitivity (24-30%), without the loss of much specificity (7-18%).

CONCLUSION

Both expert decision rules and the machine-learning algorithm performed reasonably well in identifying the majority of jobs with potential exposure to PAHs. The hybrid screening approach demonstrated that by reviewing approximately 20% of the total jobs, it could identify 87% of all jobs exposed to PAHs; sensitivity could be further increased, albeit with a decrease in specificity, by adjusting the algorithm. The resulting screening algorithm could be applied to other population-based studies of women. The process of developing the algorithm also provides a useful illustration of the strengths and potential pitfalls of these approaches to developing exposure assessment algorithms.

Inter-rater Agreement Between Exposure Assessment Using Automatic Algorithms and Using Experts

Objectives

To estimate the inter-rater agreement between exposure assessment to asthmagens in current jobs by algorithms based on task-based questionnaires (OccIDEAS) and by experts.

Methods

Participants in a cross-sectional national survey of exposure to asthmagens (AWES-Asthma) were randomly split into two subcohorts of equal size. Subcohort 1 was used to determine the most common asthmagen groups and occupational groups. From subcohort 2, a random sample of 200 participants was drawn and current occupational exposure (yes/no) was assessed in these by OccIDEAS and by two experts independently and then as a consensus. Inter-rater agreement was estimated using Cohen's Kappa coefficient. The null hypothesis was set at 0.4, because both the experts and the automatic algorithm assessed the exposure using the same task-based questionnaires and therefore an agreement better than by chance would be expected.

Results

The Kappa coefficients for the agreement between the experts and the algorithm-based assessments ranged from 0.37 to 1, while the agreement between the two experts ranged from 0.29 to 0.94, depending on the agent being assessed. After discussion by both experts the Kappa coefficients for the consensus decision and OccIDEAS were significantly larger than 0.4 for 7 of the 10 asthmagen groups, while overall the inter-rater agreement was greater than by chance (P < 0.0001).

Conclusions

The web-based application OccIDEAS is an appropriate tool for automated assessment of current exposure to asthmagens (yes/no), and requires less time-consuming work by highly-qualified research personnel than the traditional expert-based method. Further, it can learn and reuse expert determinations in future studies.

Estimation of Source-Specific Occupational Benzene Exposure in a Population-Based Case-Control Study of Non-Hodgkin Lymphoma

OBJECTIVES

Occupational exposures in population-based case-control studies are increasingly being assessed using decision rules that link participants' responses to occupational questionnaires to exposure estimates. We used a hierarchical process that incorporated decision rules and job-by-job expert review to assign occupational benzene exposure estimates in a US population-based case-control study of non-Hodgkin lymphoma.

METHODS

We conducted a literature review to identify scenarios in which occupational benzene exposure has occurred, which we grouped into 12 categories of benzene exposure sources. For each source category, we then developed decision rules for assessing probability (ordinal scale based on the likelihood of exposure > 0.02 ppm), frequency (proportion of work time exposed), and intensity of exposure (in ppm). The rules used the participants' occupational history responses and, for a subset of jobs, responses to job- and industry-specific modules. For probability and frequency, we used a hierarchical assignment procedure that prioritized subject-specific module information when available. Next, we derived job-group medians from the module responses to assign estimates to jobs with only occupational history responses. Last, we used job-by-job expert review to assign estimates when job-group medians were not available or when the decision rules identified possible heterogeneous or rare exposure scenarios. For intensity, we developed separate estimates for each benzene source category that were based on published measurement data whenever possible. Frequency and intensity annual source-specific estimates were assigned only for those jobs assigned ≥75% probability of exposure. Annual source-specific concentrations (intensity × frequency) were summed to obtain a total annual benzene concentration for each job.

RESULTS

Of the 8827 jobs reported by participants, 8% required expert review for one or more source categories. Overall, 287 (3.3%) jobs were assigned ≥75% probability of exposure from any benzene source category. The source categories most commonly assigned ≥75% probability of exposure were gasoline and degreasing. The median total annual benzene concentration among jobs assigned ≥75% probability was 0.11 ppm (interquartile range: 0.06-0.55). The highest source-specific median annual concentrations were observed for ink and printing (2.3 and 1.2 ppm, respectively).

CONCLUSIONS

The applied framework captures some subject-specific variability in work tasks, provides transparency to the exposure decision process, and facilitates future sensitivity analyses. The developed decision rules can be used as a starting point by other researchers to assess occupational benzene exposure in future population-based studies.

Use and Reliability of Exposure Assessment Methods in Occupational Case-Control Studies in the General Population: Past, Present, and Future

Introduction

Retrospective occupational exposure assessment has been challenging in case-control studies in the general population. We aimed to review (i) trends of different assessment methods used in the last 40 years and (ii) evidence of reliability for various assessment methods.

Methods

Two separate literature reviews were conducted. We first reviewed all general population cancer case-control studies published from 1975 to 2016 to summarize the exposure assessment approach used. For the second review, we systematically reviewed evidence of reliability for all methods observed in the first review.

Results

Among the 299 studies included in the first review, the most frequently used assessment methods were self-report/assessment (n = 143 studies), case-by-case expert assessment (n = 139), and job-exposure matrices (JEMs; n = 82). Usage trends for these methods remained relatively stable throughout the last four decades. Other approaches, such as the application of algorithms linking questionnaire responses to expert-assigned exposure estimates and modelling of exposure with historical measurement data, appeared in 21 studies that were published after 2000. The second review retrieved 34 comparison studies examining methodological reliability. Overall, we observed slightly higher median kappa agreement between exposure estimates from different expert assessors (~0.6) than between expert estimates and exposure estimates from self-reports (~0.5) or JEMs (~0.4). However, reported reliability measures were highly variable for different methods and agents. Limited evidence also indicates newer methods, such as assessment using algorithms and measurement-calibrated quantitative JEMs, may be as reliable as traditional methods.

Conclusion

The majority of current research assesses exposures in the population with similar methods as studies did decades ago. Though there is evidence for the development of newer approaches, more concerted effort is needed to better adopt exposure assessment methods with more transparency, reliability, and efficiency.

Using Decision Rules to Assess Occupational Exposure in Population-Based Studies

PURPOSE OF REVIEW

Population-based studies increasingly link task-based occupational questionnaire responses collected from subjects to exposure estimates via transparent, programmable decision rules. We reviewed recent applications and methodological developments of rule-based approaches.

RECENT FINDINGS

Agent-specific decision rules require interviews incorporating work-task-based questions. Some studies have developed rules before the interviews took place, while others developed rules after the interviews were completed. Agreement between rule-based estimates and exposures assigned using job-by-job expert review were generally moderate to good (Kappa = 0.4-0.8). Rules providing quantitative intensity levels using measurement data or that integrate multiple independent exposure sources for the same job represent further advances to improve the characterization of occupational exposures in population studies. Decision rules have provided transparent and reproducible assessments, reduce job-by-job review, and facilitate sensitivity analyses in epidemiologic studies. Future studies should consider the development of decision rules concurrent with the questionnaire design to facilitate occupational exposure assessment efforts.

An algorithm for quantitatively estimating non-occupational pesticide exposure intensity for spouses in the Agricultural Health Study

Residents of agricultural areas experience pesticide exposures from sources other than direct agricultural work. We developed a quantitative, active ingredient-specific algorithm for cumulative (adult, married lifetime) non-occupational pesticide exposure intensity for spouses of farmers who applied pesticides in the Agricultural Health Study (AHS). The algorithm addressed three exposure pathways: take-home, agricultural drift, and residential pesticide use. Pathway-specific equations combined (i) weights derived from previous meta-analyses of published pesticide exposure data and (ii) information from the questionnaire on frequency and duration of pesticide use by applicators, home proximity to treated fields, residential pesticide usage (e.g., termite treatments), and spouse's off-farm employment (proxy for time at home). The residential use equation also incorporated a published probability matrix that documented the likelihood active ingredients were used in home pest treatment products. We illustrate use of these equations by calculating exposure intensities for the insecticide chlorpyrifos and herbicide atrazine for 19,959 spouses. Non-zero estimates for ≥1 pathway were found for 78% and 77% of spouses for chlorpyrifos and atrazine, respectively. Variability in exposed spouses' intensity estimates was observed for both pesticides, with 75th to 25th percentile ratios ranging from 7.1 to 7.3 for take-home, 6.5 to 8.5 for drift, 2.4 to 2.8 for residential use, and 3.8 to 7.0 for the summed pathways. Take-home and drift estimates were highly correlated (≥0.98), but were not correlated with residential use (0.01‒0.02). This algorithm represents an important advancement in quantifying non-occupational pesticide relative exposure differences and will facilitate improved etiologic analyses in the AHS spouses. The algorithm could be adapted to studies with similar information.

A hybrid expert approach for retrospective assessment of occupational exposures in a population-based case-control study of cancer

BACKGROUND

While the expert-based occupational exposure assessment approach has been considered the reference method for retrospective population-based studies, its implementation in large study samples has become prohibitive. To facilitate its application and improve upon it we developed, in the context of a Montreal population-based study of prostate cancer (PROtEuS), a hybrid approach combining job-exposure profiles (JEPs) summarizing expert evaluations from previous studies and expert review. We aim to describe the hybrid expert method and its impacts on the exposures assigned in PROtEuS compared to those from a previous study coded using the traditional expert method.

METHODS

Applying the hybrid approach, experts evaluated semi-quantitative levels of confidence, concentration and frequency of exposure to 313 agents for 16,065 jobs held by 4005 subjects in PROtEuS. These assessments were compared to those from a different set of jobs coded in an earlier study of lung cancer, conducted on the same study base, for 90 blue-collar occupations and 203 agents. Endpoints evaluated included differences in the number of exposures and in the distribution of ratings across jobs, and the within-occupation variability in exposure.

RESULTS

Compared to jobs from the lung cancer study, jobs in PROtEuS had on average 0.3 more exposures. PROtEuS exposures were more often assigned definite confidence ratings, but concentration and frequency levels tended to be lower. The within-occupation variability in ratings assigned to jobs were lower in PROtEuS jobs for all metrics. This was particularly evident for concentration, although considerable variability remained with over 40% of occupation/agent cells in PROtEuS exposed at different levels. The hybrid approach reduced coding time by half, compared to the traditional expert assessment.

CONCLUSIONS

The new hybrid expert approach improved on efficiency and transparency, and resulted in greater confidence in assessments, compared to the traditional expert method applied in an earlier study involving a similar set of jobs. Assigned ratings were more homogeneous with the hybrid approach, possibly reflecting clearer guidelines for coding, greater coherence between experts and/or reliance on summaries of past assessments. Nevertheless, significant within-occupation variability remained with the hybrid approach, suggesting that experts took into account job-specific factors in their assessments.

Identifying Predictors of Stress and Job Satisfaction in a Sample of Merchant Seafarers Using Structural Equation Modeling

Background: Seafarers are amongst occupational groups with the highest risk for stress, a factor known to impact on mental health. Psychological issues such as depression, anxiety, suicide, and alcohol or drug dependence are recognized health problems within the maritime sector. The primary aim of this study was to identify which individual and occupational factors, known to impact on psychological functioning across the maritime industry and other sectors, best predict perceived stress and job satisfaction among a sample of merchant seafarers. Methods: Secondary data analysis was conducted using a work experiences and attitudes questionnaire administered by a large shipping company to seafarers within their organization. Structural equation modeling was conducted using a proposed theoretical model of perceived stress and job satisfaction in a sample of merchant seafarers. Results: While the structural equation model produced acceptable fit to the sample data according to numerous goodness-of-fit statistics, the comparative fit index and Tucker-Lewis index results indicated less than satisfactory model fit. The model explained 23.8% of variance in the criterion variable of perceived stress, and the strongest predictive effect was for dispositional resilience. The model explained 70.6% of variance in the criterion variable of job satisfaction, and the strongest predictive effect was for instrumental work support. Conclusion: When addressing the psychosocial well-being of merchant seafarers, findings of this study suggest that dispositional resilience may be a particularly important factor with regards to perceived stress, while instrumental work support appears to be a critical factor in relation to job satisfaction. Importantly, however, an overall work environment that is perceived by employees as supportive, equal and just is a cornerstone for the psychosocial well-being of seafarers.

Decision rule approach applied to estimate occupational lead exposure in a case-control study of kidney cancer

BACKGROUND

We developed a systematic, data-driven approach to estimate metrics of occupational exposure to lead to aid in epidemiologic analyses in a case-control study of kidney cancer.

METHODS

Probability of exposure to ten lead sources was assigned using decision rules developed from an extensive literature review and expert judgement. For jobs with >50% probability of exposure, we assigned source-specific frequency based on subjects' self-reported task frequencies or means of subjects' job-groups and source-specific intensity estimates of blood lead (μg/dL).

RESULTS

In our study, 18.7% of employed person-years were associated with high (≥80%) probability of exposure to any lead source. The most common medium (>50%) or high probability source of lead exposure was leaded gasoline (2.5% and 11.5% of employed person-years, respectively). The median blood lead attributed to occupational exposure was 3.1 μg/dL.

CONCLUSIONS

These rules can aid in future studies after population-specific adaption for geographic differences and different exposure scenarios.

Retrospective Assessment of Occupational Exposures for the GENEVA Study of ALS among Military Veterans

Objective

This paper describes the retrospective exposure assessment conducted to assess occupational exposures for the Genes and Environmental Exposures in Veterans (GENEVA) study, a case-control study investigating the joint contribution of genetics and environmental exposures to the risk of amyotrophic lateral sclerosis (ALS) among military veterans.

Methods

Occupational histories for 1597 study participants collected as part of the GENEVA study were the basis for this retrospective exposure assessment. The data set included 15528 jobs held from 1924 to 2010, representing 4539 unique industry and occupation (I&O) combinations. Three industrial hygiene experts were recruited to independently rate occupational exposures to specific agents previously associated with an increased risk of ALS. Utilizing information on industry, job title, tasks performed, and materials used for each job held, raters assigned exposures associated with each I&O for the 'current time' defined as the period after 1995 (post-1995). The exposure assessment targeted agents identified as potential occupational risk factors for ALS. Experts rated semi-quantitatively exposure intensity in five exposure categories (0-4) for Group A agents (lead, formaldehyde, hydrocarbon solvents, and chlorinated solvents) and qualitatively as yes/no (1/0) exposed for Group B agents (mercury, selenium, arsenic, polychlorinated biphenyls, electromagnetic field, pesticides, and viral agents). Confidence scores (0-3) were reported for every I&O rated based on raters' experience with that industry and/or job. Each I&O was assigned an average exposure score of the raters and an alternative exposure rating was developed for each I&O by excluding low confidence (<2) scores before averaging. Exposure reconstruction for jobs held pre-1995 was done by comparing exposure data extracted from the OSHA Chemical Exposure and Health Database (CEHD) during pre-1995 and post-1995. For agents with limited exposure data in the CEHD, pre-1995 exposures were determined based on raters' judgment.

Results

The proportion of I&O combinations determined to be 'exposed' ranged from 0.1 to 26% across different agents, with the highest values corresponding to hydrocarbon solvents and the lowest to selenium. Industries with the highest proportion of exposed records include manufacturing, utilities, healthcare, and military with non-combat jobs. Analyses for raters' reliability showed the best agreement between the raters when rating exposure to viral agents (kappa = 0.67), hydrocarbon solvents (kappa = 0.53), and lead (kappa = 0.50). The proportion of 'exposed' I&O combinations increased for hydrocarbon solvents, chlorinated solvents, and pesticides when exposure ratings were adjusted by raters' confidence. Compared to post-1995, exposures in the earlier period (pre-1995) were deemed higher or the same for most of the agents and lower for formaldehyde and electromagnetic field exposures.

Conclusions

Our results indicate that using raters' confidence assessment in determining exposure scores increases both the proportion of I&O combinations regarded as exposed and the intensity scores, suggesting raters tend to be conservative in their assessment when they lack detailed knowledge of an industry or job.

Evaluation of Automatically Assigned Job-Specific Interview Modules

OBJECTIVE

In community-based epidemiological studies, job- and industry-specific 'modules' are often used to systematically obtain details about the subject's work tasks. The module assignment is often made by the interviewer, who may have insufficient occupational hygiene knowledge to assign the correct module. We evaluated, in the context of a case-control study of lymphoid neoplasms in Asia ('AsiaLymph'), the performance of an algorithm that provided automatic, real-time module assignment during a computer-assisted personal interview.

METHODS

AsiaLymph's occupational component began with a lifetime occupational history questionnaire with free-text responses and three solvent exposure screening questions. To assign each job to one of 23 study-specific modules, an algorithm automatically searched the free-text responses to the questions 'job title' and 'product made or services provided by employer' using a list of module-specific keywords, comprising over 5800 keywords in English, Traditional and Simplified Chinese. Hierarchical decision rules were used when the keyword match triggered multiple modules. If no keyword match was identified, a generic solvent module was assigned if the subject responded 'yes' to any of the three solvent screening questions. If these question responses were all 'no', a work location module was assigned, which redirected the subject to the farming, teaching, health professional, solvent, or industry solvent modules or ended the questions for that job, depending on the location response. We conducted a reliability assessment that compared the algorithm-assigned modules to consensus module assignments made by two industrial hygienists for a subset of 1251 (of 11409) jobs selected using a stratified random selection procedure using module-specific strata. Discordant assignments between the algorithm and consensus assignments (483 jobs) were qualitatively reviewed by the hygienists to evaluate the potential information lost from missed questions with using the algorithm-assigned module (none, low, medium, high).

RESULTS

The most frequently assigned modules were the work location (33%), solvent (20%), farming and food industry (19%), and dry cleaning and textile industry (6.4%) modules. In the reliability subset, the algorithm assignment had an exact match to the expert consensus-assigned module for 722 (57.7%) of the 1251 jobs. Overall, adjusted for the proportion of jobs in each stratum, we estimated that 86% of the algorithm-assigned modules would result in no information loss, 2% would have low information loss, and 12% would have medium to high information loss. Medium to high information loss occurred for <10% of the jobs assigned the generic solvent module and for 21, 32, and 31% of the jobs assigned the work location module with location responses of 'someplace else', 'factory', and 'don't know', respectively. Other work location responses had ≤8% with medium to high information loss because of redirections to other modules. Medium to high information loss occurred more frequently when a job description matched with multiple keywords pointing to different modules (29-69%, depending on the triggered assignment rule).

CONCLUSIONS

These evaluations demonstrated that automatically assigned modules can reliably reproduce an expert's module assignment without the direct involvement of an industrial hygienist or interviewer. The feasibility of adapting this framework to other studies will be language- and exposure-specific.

Assessment of occupational risks to extremely low frequency magnetic fields: Validation of an empirical non-expert approach

The expert method of exposure assignment involves relying on chemists or hygienists to estimate occupational exposures using information collected on study subjects. Once the estimation method for a particular contaminant has been made available in the literature, it is not known whether a non-expert, briefly trained by an expert remaining available to answer ad hoc questions, can provide reliable exposure estimates. We explored this issue by comparing estimates of exposure to extremely low frequency magnetic fields (ELF-MF) obtained by an expert to those from a non-expert. Using a published exposure matrix, both the expert and non-expert independently calculated a weekly time-weighted average exposure for 208 maternal jobs by considering three main determinants: the work environment, magnetic field sources, and duration of use or exposure to given sources. Agreement between assessors was tested using the Bland-Altman 95% limits of agreement. The overall mean difference in estimates between the expert and non-expert was 0.004 μT (standard deviation 0.104). The 95% limits of agreement were − 0.20 μT and + 0.21 μT. The work environments and exposure sources were almost always similarly identified but there were differences in estimating exposure duration. This occurred mainly when information collected from study subjects was not sufficiently detailed. Our results suggest that following a short training period and the availability of a clearly described method for estimating exposures, a non-expert can cost-efficiently and reliably assign exposure, at least to ELF-MF.

•

Retrospective occupational exposure assessment often relies on the expert method.

•

Using a published job-exposure matrix, a trained non-expert can correctly estimate an individual's specific exposure.

•

Non-expert method is a feasible, practical, and based on our study, a valid approach to code exposure.

Wishful Thinking? Inside the Black Box of Exposure Assessment

BACKGROUND

Decision-making processes used by experts when undertaking occupational exposure assessment are relatively unknown, but it is often assumed that there is a common underlying method that experts employ. However, differences in training and experience of assessors make it unlikely that one general method for expert assessment would exist. Therefore, there are concerns about formalizing, validating, and comparing expert estimates within and between studies that are difficult, if not impossible, to characterize. Heuristics on the other hand (the processes involved in decision making) have been extensively studied. Heuristics are deployed by everyone as short-cuts to make the often complex process of decision-making simpler, quicker, and less burdensome. Experts' assessments are often subject to various simplifying heuristics as a way to reach a decision in the absence of sufficient data. Therefore, investigating the underlying heuristics or decision-making processes involved may help to shed light on the 'black box' of exposure assessment.

METHODS

A mixed method study was conducted utilizing both a web-based exposure assessment exercise incorporating quantitative and semiqualitative elements of data collection, and qualitative semi-structured interviews with exposure assessors. Qualitative data were analyzed using thematic analysis.

RESULTS

Twenty-five experts completed the web-based exposure assessment exercise and 8 of these 25 were randomly selected to participate in the follow-up interview. Familiar key themes relating to the exposure assessment exercise emerged; 'intensity'; 'probability'; 'agent'; 'process'; and 'duration' of exposure. However, an important aspect of the detailed follow-up interviews revealed a lack of structure and order with which participants described their decision making. Participants mostly described some form of an iterative process, heavily relying on the anchoring and adjustment heuristic, which differed between experts.

CONCLUSION

In spite of having undertaken comparable training (in occupational hygiene or exposure assessment), experts use different methods to assess exposure. Decision making appears to be an iterative process with heavy reliance on the key heuristic of anchoring and adjustment. Using multiple experts to assess exposure while providing some form of anchoring scenario to build from, and additional training in understanding the impact of simple heuristics on the process of decision making, is likely to produce a more methodical approach to assessment; thereby improving consistency and transparency in expert exposure assessment.

Combining Decision Rules from Classification Tree Models and Expert Assessment to Estimate Occupational Exposure to Diesel Exhaust for a Case-Control Study

OBJECTIVES

To efficiently and reproducibly assess occupational diesel exhaust exposure in a Spanish case-control study, we examined the utility of applying decision rules that had been extracted from expert estimates and questionnaire response patterns using classification tree (CT) models from a similar US study.

METHODS

First, previously extracted CT decision rules were used to obtain initial ordinal (0-3) estimates of the probability, intensity, and frequency of occupational exposure to diesel exhaust for the 10 182 jobs reported in a Spanish case-control study of bladder cancer. Second, two experts reviewed the CT estimates for 350 jobs randomly selected from strata based on each CT rule's agreement with the expert ratings in the original study [agreement rate, from 0 (no agreement) to 1 (perfect agreement)]. Their agreement with each other and with the CT estimates was calculated using weighted kappa (κ w) and guided our choice of jobs for subsequent expert review. Third, an expert review comprised all jobs with lower confidence (low-to-moderate agreement rates or discordant assignments, n = 931) and a subset of jobs with a moderate to high CT probability rating and with moderately high agreement rates (n = 511). Logistic regression was used to examine the likelihood that an expert provided a different estimate than the CT estimate based on the CT rule agreement rates, the CT ordinal rating, and the availability of a module with diesel-related questions.

RESULTS

Agreement between estimates made by two experts and between estimates made by each of the experts and the CT estimates was very high for jobs with estimates that were determined by rules with high CT agreement rates (κ w: 0.81-0.90). For jobs with estimates based on rules with lower agreement rates, moderate agreement was observed between the two experts (κ w: 0.42-0.67) and poor-to-moderate agreement was observed between the experts and the CT estimates (κ w: 0.09-0.57). In total, the expert review of 1442 jobs changed 156 probability estimates, 128 intensity estimates, and 614 frequency estimates. The expert was more likely to provide a different estimate when the CT rule agreement rate was <0.8, when the CT ordinal ratings were low to moderate, or when a module with diesel questions was available.

CONCLUSIONS

Our reliability assessment provided important insight into where to prioritize additional expert review; as a result, only 14% of the jobs underwent expert review, substantially reducing the exposure assessment burden. Overall, we found that we could efficiently, reproducibly, and reliably apply CT decision rules from one study to assess exposure in another study.

Using hierarchical cluster models to systematically identify groups of jobs with similar occupational questionnaire response patterns to assist rule-based expert exposure assessment in population-based studies

OBJECTIVES

Rule-based expert exposure assessment based on questionnaire response patterns in population-based studies improves the transparency of the decisions. The number of unique response patterns, however, can be nearly equal to the number of jobs. An expert may reduce the number of patterns that need assessment using expert opinion, but each expert may identify different patterns of responses that identify an exposure scenario. Here, hierarchical clustering methods are proposed as a systematic data reduction step to reproducibly identify similar questionnaire response patterns prior to obtaining expert estimates. As a proof-of-concept, we used hierarchical clustering methods to identify groups of jobs (clusters) with similar responses to diesel exhaust-related questions and then evaluated whether the jobs within a cluster had similar (previously assessed) estimates of occupational diesel exhaust exposure.

METHODS

Using the New England Bladder Cancer Study as a case study, we applied hierarchical cluster models to the diesel-related variables extracted from the occupational history and job- and industry-specific questionnaires (modules). Cluster models were separately developed for two subsets: (i) 5395 jobs with ≥1 variable extracted from the occupational history indicating a potential diesel exposure scenario, but without a module with diesel-related questions; and (ii) 5929 jobs with both occupational history and module responses to diesel-relevant questions. For each subset, we varied the numbers of clusters extracted from the cluster tree developed for each model from 100 to 1000 groups of jobs. Using previously made estimates of the probability (ordinal), intensity (µg m(-3) respirable elemental carbon), and frequency (hours per week) of occupational exposure to diesel exhaust, we examined the similarity of the exposure estimates for jobs within the same cluster in two ways. First, the clusters' homogeneity (defined as >75% with the same estimate) was examined compared to a dichotomized probability estimate (<5 versus ≥5%; <50 versus ≥50%). Second, for the ordinal probability metric and continuous intensity and frequency metrics, we calculated the intraclass correlation coefficients (ICCs) between each job's estimate and the mean estimate for all jobs within the cluster.

RESULTS

Within-cluster homogeneity increased when more clusters were used. For example, ≥80% of the clusters were homogeneous when 500 clusters were used. Similarly, ICCs were generally above 0.7 when ≥200 clusters were used, indicating minimal within-cluster variability. The most within-cluster variability was observed for the frequency metric (ICCs from 0.4 to 0.8). We estimated that using an expert to assign exposure at the cluster-level assignment and then to review each job in non-homogeneous clusters would require ~2000 decisions per expert, in contrast to evaluating 4255 unique questionnaire patterns or 14983 individual jobs.

CONCLUSIONS

This proof-of-concept shows that using cluster models as a data reduction step to identify jobs with similar response patterns prior to obtaining expert ratings has the potential to aid rule-based assessment by systematically reducing the number of exposure decisions needed. While promising, additional research is needed to quantify the actual reduction in exposure decisions and the resulting homogeneity of exposure estimates within clusters for an exposure assessment effort that obtains cluster-level expert assessments as part of the assessment process.

Comparison of ordinal and nominal classification trees to predict ordinal expert-based occupational exposure estimates in a case-control study

OBJECTIVES

To evaluate occupational exposures in case-control studies, exposure assessors typically review each job individually to assign exposure estimates. This process lacks transparency and does not provide a mechanism for recreating the decision rules in other studies. In our previous work, nominal (unordered categorical) classification trees (CTs) generally successfully predicted expert-assessed ordinal exposure estimates (i.e. none, low, medium, high) derived from occupational questionnaire responses, but room for improvement remained. Our objective was to determine if using recently developed ordinal CTs would improve the performance of nominal trees in predicting ordinal occupational diesel exhaust exposure estimates in a case-control study.

METHODS

We used one nominal and four ordinal CT methods to predict expert-assessed probability, intensity, and frequency estimates of occupational diesel exhaust exposure (each categorized as none, low, medium, or high) derived from questionnaire responses for the 14983 jobs in the New England Bladder Cancer Study. To replicate the common use of a single tree, we applied each method to a single sample of 70% of the jobs, using 15% to test and 15% to validate each method. To characterize variability in performance, we conducted a resampling analysis that repeated the sample draws 100 times. We evaluated agreement between the tree predictions and expert estimates using Somers' d, which measures differences in terms of ordinal association between predicted and observed scores and can be interpreted similarly to a correlation coefficient.

RESULTS

From the resampling analysis, compared with the nominal tree, an ordinal CT method that used a quadratic misclassification function and controlled tree size based on total misclassification cost had a slightly better predictive performance that was statistically significant for the frequency metric (Somers' d: nominal tree = 0.61; ordinal tree = 0.63) and similar performance for the probability (nominal = 0.65; ordinal = 0.66) and intensity (nominal = 0.65; ordinal = 0.65) metrics. The best ordinal CT predicted fewer cases of large disagreement with the expert assessments (i.e. no exposure predicted for a job with high exposure and vice versa) compared with the nominal tree across all of the exposure metrics. For example, the percent of jobs with expert-assigned high intensity of exposure that the model predicted as no exposure was 29% for the nominal tree and 22% for the best ordinal tree.

CONCLUSIONS

The overall agreements were similar across CT models; however, the use of ordinal models reduced the magnitude of the discrepancy when disagreements occurred. As the best performing model can vary by situation, researchers should consider evaluating multiple CT methods to maximize the predictive performance within their data.

Reliability and validity of expert assessment based on airborne and urinary measures of nickel and chromium exposure in the electroplating industry

The reliability and validity of six experts' exposure ratings were evaluated for 64 nickel-exposed and 72 chromium-exposed workers from six Shanghai electroplating plants based on airborne and urinary nickel and chromium measurements. Three industrial hygienists and three occupational physicians independently ranked the exposure intensity of each metal on an ordinal scale (1-4) for each worker's job in two rounds: the first round was based on responses to an occupational history questionnaire and the second round also included responses to an electroplating industry-specific questionnaire. The Spearman correlation (r(s)) was used to compare each rating's validity to its corresponding subject-specific arithmetic mean of four airborne or four urinary measurements. Reliability was moderately high (weighted kappa range=0.60-0.64). Validity was poor to moderate (r(s)=-0.37-0.46) for both airborne and urinary concentrations of both metals. For airborne nickel concentrations, validity differed by plant. For dichotomized metrics, sensitivity and specificity were higher based on urinary measurements (47-78%) than airborne measurements (16-50%). Few patterns were observed by metal, assessment round, or expert type. These results suggest that, for electroplating exposures, experts can achieve moderately high agreement and (reasonably) distinguish between low and high exposures when reviewing responses to in-depth questionnaires used in population-based case-control studies.

Developing estimates of frequency and intensity of exposure to three types of metalworking fluids in a population-based case-control study of bladder cancer

BACKGROUND

A systematic, transparent, and data-driven approach was developed to estimate frequency and intensity of exposure to straight, soluble, and synthetic/semi-synthetic metalworking fluids (MWFs) within a case-control study of bladder cancer in New England.

METHODS

We assessed frequency using individual-level information from job-specific questionnaires wherever possible, then derived and applied job group-level patterns to likely exposed jobs with less information. Intensity estimates were calculated using a statistical model developed from measurements and determinants extracted from the published literature.

RESULTS

For jobs with probabilities of exposure≥0.5, median frequencies were 8-10 hr/week, depending on MWF type. Median intensities for these jobs were 2.5, 2.1, and 1.0 mg/m3 for soluble, straight, and synthetic/semi-synthetic MWFs, respectively.

CONCLUSIONS

Compared to case-by-case assessment, these data-driven decision rules are transparent and reproducible and may result in less biased estimates. These rules can also aid future exposure assessments of MWFs in population-based studies.

Systematically extracting metal- and solvent-related occupational information from free-text responses to lifetime occupational history questionnaires

OBJECTIVES

Lifetime occupational history (OH) questionnaires often use open-ended questions to capture detailed information about study participants' jobs. Exposure assessors use this information, along with responses to job- and industry-specific questionnaires, to assign exposure estimates on a job-by-job basis. An alternative approach is to use information from the OH responses and the job- and industry-specific questionnaires to develop programmable decision rules for assigning exposures. As a first step in this process, we developed a systematic approach to extract the free-text OH responses and convert them into standardized variables that represented exposure scenarios.

METHODS

Our study population comprised 2408 subjects, reporting 11991 jobs, from a case-control study of renal cell carcinoma. Each subject completed a lifetime OH questionnaire that included verbatim responses, for each job, to open-ended questions including job title, main tasks and activities (task), tools and equipment used (tools), and chemicals and materials handled (chemicals). Based on a review of the literature, we identified exposure scenarios (occupations, industries, tasks/tools/chemicals) expected to involve possible exposure to chlorinated solvents, trichloroethylene (TCE) in particular, lead, and cadmium. We then used a SAS macro to review the information reported by study participants to identify jobs associated with each exposure scenario; this was done using previously coded standardized occupation and industry classification codes, and a priori lists of associated key words and phrases related to possibly exposed tasks, tools, and chemicals. Exposure variables representing the occupation, industry, and task/tool/chemicals exposure scenarios were added to the work history records of the study respondents. Our identification of possibly TCE-exposed scenarios in the OH responses was compared to an expert's independently assigned probability ratings to evaluate whether we missed identifying possibly exposed jobs.

RESULTS

Our process added exposure variables for 52 occupation groups, 43 industry groups, and 46 task/tool/chemical scenarios to the data set of OH responses. Across all four agents, we identified possibly exposed task/tool/chemical exposure scenarios in 44-51% of the jobs in possibly exposed occupations. Possibly exposed task/tool/chemical exposure scenarios were found in a nontrivial 9-14% of the jobs not in possibly exposed occupations, suggesting that our process identified important information that would not be captured using occupation alone. Our extraction process was sensitive: for jobs where our extraction of OH responses identified no exposure scenarios and for which the sole source of information was the OH responses, only 0.1% were assessed as possibly exposed to TCE by the expert.

CONCLUSIONS

Our systematic extraction of OH information found useful information in the task/chemicals/tools responses that was relatively easy to extract and that was not available from the occupational or industry information. The extracted variables can be used as inputs in the development of decision rules, especially for jobs where no additional information, such as job- and industry-specific questionnaires, is available.

Inside the black box: starting to uncover the underlying decision rules used in a one-by-one expert assessment of occupational exposure in case-control studies

OBJECTIVES

Evaluating occupational exposures in population-based case-control studies often requires exposure assessors to review each study participant's reported occupational information job-by-job to derive exposure estimates. Although such assessments likely have underlying decision rules, they usually lack transparency, are time consuming and have uncertain reliability and validity. We aimed to identify the underlying rules to enable documentation, review and future use of these expert-based exposure decisions.

METHODS

Classification and regression trees (CART, predictions from a single tree) and random forests (predictions from many trees) were used to identify the underlying rules from the questionnaire responses, and an expert's exposure assignments for occupational diesel exhaust exposure for several metrics: binary exposure probability and ordinal exposure probability, intensity and frequency. Data were split into training (n=10 488 jobs), testing (n=2247) and validation (n=2248) datasets.

RESULTS

The CART and random forest models' predictions agreed with 92-94% of the expert's binary probability assignments. For ordinal probability, intensity and frequency metrics, the two models extracted decision rules more successfully for unexposed and highly exposed jobs (86-90% and 57-85%, respectively) than for low or medium exposed jobs (7-71%).

CONCLUSIONS

CART and random forest models extracted decision rules and accurately predicted an expert's exposure decisions for the majority of jobs, and identified questionnaire response patterns that would require further expert review if the rules were applied to other jobs in the same or different study. This approach makes the exposure assessment process in case-control studies more transparent, and creates a mechanism to efficiently replicate exposure decisions in future studies.

Parental occupational exposure to engine exhausts and childhood brain tumors

Childhood brain tumors (CBT) are the leading cause of cancer death in children; their risk factors are still largely unknown. Since most CBTs are diagnosed before five years of age, prenatal exposure and early postnatal factors may be involved in their etiology. We investigated the association between CBT and parental occupational exposure to engine exhausts in an Australian population-based case-control study. Parents of 306 cases and 950 controls completed detailed occupational histories. Odds ratios (OR) and 95% confidence intervals (CI) were estimated for both maternal and paternal exposure in key time periods. Increased risks were observed for maternal exposure to diesel exhaust any time before the child's birth (OR 2.03, 95% CI 1.09-3.81) and paternal exposure around the time of the child's conception (OR 1.62, 95% CI 1.12-2.34). No clear associations with other engine exhausts were found. Our results suggest that parental occupational exposure to diesel exhaust may increase the risk of CBT.

Comparison of algorithm-based estimates of occupational diesel exhaust exposure to those of multiple independent raters in a population-based case-control study

OBJECTIVES

Algorithm-based exposure assessments based on patterns in questionnaire responses and professional judgment can readily apply transparent exposure decision rules to thousands of jobs quickly. However, we need to better understand how algorithms compare to a one-by-one job review by an exposure assessor. We compared algorithm-based estimates of diesel exhaust exposure to those of three independent raters within the New England Bladder Cancer Study, a population-based case-control study, and identified conditions under which disparities occurred in the assessments of the algorithm and the raters.

METHODS

Occupational diesel exhaust exposure was assessed previously using an algorithm and a single rater for all 14 983 jobs reported by 2631 study participants during personal interviews conducted from 2001 to 2004. Two additional raters independently assessed a random subset of 324 jobs that were selected based on strata defined by the cross-tabulations of the algorithm and the first rater's probability assessments for each job, oversampling their disagreements. The algorithm and each rater assessed the probability, intensity and frequency of occupational diesel exhaust exposure, as well as a confidence rating for each metric. Agreement among the raters, their aggregate rating (average of the three raters' ratings) and the algorithm were evaluated using proportion of agreement, kappa and weighted kappa (κw). Agreement analyses on the subset used inverse probability weighting to extrapolate the subset to estimate agreement for all jobs. Classification and Regression Tree (CART) models were used to identify patterns in questionnaire responses that predicted disparities in exposure status (i.e., unexposed versus exposed) between the first rater and the algorithm-based estimates.

RESULTS

For the probability, intensity and frequency exposure metrics, moderate to moderately high agreement was observed among raters (κw = 0.50-0.76) and between the algorithm and the individual raters (κw = 0.58-0.81). For these metrics, the algorithm estimates had consistently higher agreement with the aggregate rating (κw = 0.82) than with the individual raters. For all metrics, the agreement between the algorithm and the aggregate ratings was highest for the unexposed category (90-93%) and was poor to moderate for the exposed categories (9-64%). Lower agreement was observed for jobs with a start year <1965 versus ≥1965. For the confidence metrics, the agreement was poor to moderate among raters (κw = 0.17-0.45) and between the algorithm and the individual raters (κw = 0.24-0.61). CART models identified patterns in the questionnaire responses that predicted a fair-to-moderate (33-89%) proportion of the disagreements between the raters' and the algorithm estimates.

DISCUSSION

The agreement between any two raters was similar to the agreement between an algorithm-based approach and individual raters, providing additional support for using the more efficient and transparent algorithm-based approach. CART models identified some patterns in disagreements between the first rater and the algorithm. Given the absence of a gold standard for estimating exposure, these patterns can be reviewed by a team of exposure assessors to determine whether the algorithm should be revised for future studies.