Public health implications of the variability in the interpretation of 'B' readings for pleural changes

Pneumoconioses Radiographs in a Large Population of U.S. Coal Workers: Variability in A Reader and B Reader Classifications by Using the International Labour Office Classification

Purpose To assess the level of concordance between chest radiographic classifications of A and B Readers in a national surveillance program offered to U.S. coal miners over an approximate 36-year period. Materials and Methods The National Institute for Occupational Safety and Health (NIOSH) Coal Workers' Health Surveillance Program (CWHSP) is a surveillance program with nonresearch designation and is exempt from Human Subjects Review Board approval (11-DRDS-NR03). Thirty-six years of data (1979-2015) from the CWHSP were analyzed, which included all conventional screen-film radiographs with a classification by at least one A Reader and one B Reader. Agreement was assessed by using κ statistics; prevalence ratios were used to describe differences between A and B Reader determinations of image technical quality, small opacity profusion, and presence of large opacities and pleural abnormalities. Results The analysis included 79 185 matched A and B Reader chest radiograph classifications. A majority of both A and B Readers were radiologists (74.2% [213 of 287] vs 64.7% [22 of 34]; P = .04). A and B Readers had minimal agreement on technical image quality (κ = 0.0796; 95% confidence interval [CI]: 0.07, 0.08) and the distribution of small opacity profusion (subcategory κ, 0.2352; 95% CI: 0.22, 0.25). A Readers classified more images as "good" quality (prevalence ratio, 1.38; 95% CI: 1.35, 1.41) and identified more pneumoconiosis (prevalence ratio, 1.22; 95% CI: 1.20, 1.23). Conclusion A Readers classified substantially more radiographs with evidence of pneumoconiosis and classified higher small opacity profusion compared with B Readers. These observations reinforce the importance of multiple classifications by readers who have demonstrated ongoing competence in the International Labour Office classification system to ensure accurate radiographic classifications. ^© RSNA, 2017.

Changes in B-Readings Over Time in the United States Navy Asbestos Medical Surveillance Program

OBJECTIVE

To study the impact of variability among B-readers on clinical occupational medicine.

METHODS

A total of 419,770 B-reading reports from February 1980 to May 2004 in the US Navy Asbestos Medical Surveillance Program were analyzed for changes in category from the previous B-reading on each individual worker.

RESULTS

Over 7% of films were categorized as worse (ie, read as going from negative to positive), and over 6% were categorized as better. When profusion categories were reported as different from the previous reading (over 6% of the time), they were more frequently read as 2 or more minor categories worse or better.

CONCLUSIONS

Changes from previous B-readings are common, and may have clinical and other implications, which are discussed. B-readings should not be used as the sole basis for determining the presence or absence of pneumoconiosis.

Asbestos-related pleural diseases: dimensional criteria are not appropriate to differentiate diffuse pleural thickening from pleural plaques

BACKGROUND

In the literature, the criteria used to define pleural plaques (PP) and diffuse pleural thickening (DPT) are very heterogeneous and often imprecise. A multicenter restropective study was conducted to assess the relevance of two radiographic definitions of DPT.

METHODS

The study population consisted of 287 subjects with asbestos-related pleural thickening. Two definitions were used to characterize DPT on postero-anterior chest radiographs: definition 1: pleural thickening associated with obliteration of the costophrenic angle; definition 2: pleural thickening at least 5 mm wide, extending for more than one quarter of the chest wall. Prevalence of respiratory symptoms and pulmonary function tests were compared in the DPT and PP groups resulting from the two definitions of DPT.

RESULTS

According to definition 1, 34 patients (11.8%) were classified in the DPT group. Prevalence of chronic sputum, dyspnea, and chest pain was significantly higher in this group than in the PP group. FEV(1), FVC, and TLC were significantly lower. The differences persisted after adjustment for confounding factors. According to definition 2,102 patients (36.6%) were classified in the DPT group. DPT and PP groups did not differ in terms of prevalence of respiratory symptoms, or pulmonary function tests. Agreement between readers was significantly better when using definition 1.

CONCLUSIONS

Obliteration of costophrenic angle is a much more reliable sign than dimensional criteria to characterize DPT.

International Labor Office Classification System in the age of imaging: relevant or redundant

The 1980 International Labor Office International Classification of Radiographs of Pneumoconioses is a widely used epidemiologic tool with a storied past. This article reviews its development and examines its applications to occupational lung disease and the controversies generated in that process. The question of its relevancy to current imaging practices is discussed.

Reliability and validity of chest radiograph surveillance programs

STUDY OBJECTIVES

Due to the lack of consensus in the literature in the use of posteroanterior (PA) vs PA with right and left oblique views as the optimum radiograph surveillance methodology to investigate pleural changes, a study was undertaken to evaluate the reliability, sensitivity, and specificity of these two approaches.

DESIGN

Three experienced radiologist B readers used the 1980 International Labor Office classification system for pneumoconiosis to independently read chest radiographs of workers with individual identifiers masked. All radiographs were read first as a PA view only. Unknown to the B readers, each subject's PA was then matched to his or her corresponding right and left oblique views (film triad) and re-read several weeks later.

SETTING AND PARTICIPANTS

The respiratory health of 652 workers exposed to refractory ceramic fiber was assessed as part of cross-sectional and longitudinal surveillance programs.

MEASUREMENTS AND RESULTS

kappa Statistics for interreader and intrareader reliability between the PA view and film triad methods were calculated. Sensitivity, specificity, and positive predictive value were assessed by comparing the initial cross-sectional study to the longitudinal study. The film triad method had considerably higher interreader reliability (kappa = 0.59) compared to the PA-only method (kappa = 0.44). Results from the initial cross-sectional study were then compared to findings evaluated longitudinally. The film triad again was superior, demonstrating a positive predictive value of 73.7% compared to only 47.8% for the PA method.

CONCLUSIONS

It is our recommendation that the film triad method be used in surveillance studies where both parenchymal and pleural changes are anticipated.

Observer variation in computed tomography of pleural lesions in subjects exposed to indoor asbestos

To assess the reliability of computed tomography (CT) in detecting discrete pleural lesions, the interobserver and intra-observer variability in reading the conventional and high-resolution CT (HRCT) scans of 100 volunteers, who had worked for > or = 10 yrs in a building with known asbestos contamination, was evaluated. In the first session, pleural abnormalities were detected by a single radiologist (A1) in 13 subjects. In the second session, the scans were read again independently by the same radiologist (A2) and two other experienced radiologists (B, C). The final decision for the presence of pleural lesions was made in a final consensus reading. This gave a diagnosis of pleural abnormalities in 18 subjects, of whom eight (44%) had been detected by all three readers, five (28%) by two readers and four (22%) by only one reader; one scan, rated normal by all readers during the second session, was reconsidered because pleural abnormalities had been noted at the first reading (A1). The intra-observer agreement for reader A was good (kappa (kappa) 0.68) but the interobserver agreement between the readers was only fair to moderate (weighted kappa: A2-B=0.43, A2-C = 0.45, B-C = 0.26) in the second reading session. In conclusion, when looking for the prevalence of pleural lesions in indoor asbestos exposed subjects, the potential lack of consistency in reporting the presence of small pleural abnormalities must be borne in mind and strict precautions must be taken.

Prevalence of small lung opacities in populations unexposed to dusts. A literature analysis

OBJECTIVES

Despite the wide use of the International Labor Organization (ILO) system for reading chest radiographs, little information is available regarding the prevalence of abnormalities in populations unexposed to dusts. Prevalence studies of radiographic changes consistent with dust inhalation, as classified by the system, would be more meaningful if there were better understanding regarding the extent of abnormalities in unexposed populations.

DESIGN

To determine small opacity prevalence in unexposed populations, a review of articles published since 1970 that used the ILO system to classify radiographs of the unexposed, either as subjects or control subjects, was performed. Criteria for inclusion in this review included ascertainment of the lack of exposure of subjects to occupational dusts, and independent reading of radiographs by at least two readers certified in the ILO system (B readers) or experienced in its use. A total of eight published articles presenting data on nine study populations were included in this study.

RESULTS

The prevalence of small opacities graded 1/0 or greater varied widely, with a range from 0.21 to 11.7%. A meta-analysis of the published data yielded a population prevalence of 5.3% (95% confidence interval [CI] = 2.9 to 7.7%). The prevalence was significantly greater in Europe than in North America (Europe, 11.3%; 95% CI = 10.1 to 12.5%; North America, 1.6%; 95% CI = 0.6 to 2.6%). A subset of the studies contained information on gender that showed greater prevalence of lung opacities in male subjects than female subjects (male subjects, 5.5%; 95% CI = 3.4 to 7.6%; female subjects, 3.5%; 95% CI = 1.3 to 5.8%). Based on estimated age information, the studies were divided into two strata (mean age < 50 years vs > or = 50 years). The age-specific pooled prevalence was higher in the studies with mean age > or = 50 years than studies with mean age < 50 years in both Europe (11.7% vs 9.6%) and North America (2.3% vs 0.6%). Prevalence of lung opacities remained significantly higher in Europe and North America in each age stratum. The large difference in the prevalence between Europe and North America could not be explained on the basis of age, gender, or smoking history, although available age and smoking data are less robust.

CONCLUSIONS

These results indicate that a background level of opacities consistent with the radiographic appearance of pneumoconiosis exists in populations considered to be free of occupational dust exposure. Environmental and unaccounted occupational exposures, as well as reader variability, all may play a role in the determination of small opacity prevalence in these subjects and may explain the large differences between Europe and North America. Thorough ascertainments of occupational and environmental exposures are essential to determine the true significance of opacities in populations who are not exposed to dust.

Prevalence of radiographic small lung opacities and pleural abnormalities in a representative adult population sample

The prevalence of radiographic widespread small lung opacities and pleural abnormalities in the general population was assessed as part of the Mini-Finland Health Survey, which was based on a representative sample of people aged 30 years or over. Full-size chest radiographs of 7,095 persons (89% of the sample) were classified. Two radiologists classified all the radiographs according to the ILO 1980 Classification of Radiographs of Pneumoconioses. Small lung opacities and pleural abnormalities were considerably more prevalent in the older age groups and in men. Small lung opacities on the radiographs and pleural abnormalities were associated with work in industrial occupations, particularly in men. There was a clear association between small lung opacities and pleural abnormalities. The sex differences may be related to occupational factors, particularly dust exposure. The elevated risk of pleural plaques in persons with small lung opacities on the radiographs may indicate a fiber effect.

Use of chest radiographs in epidemiological investigations of pneumoconioses

The International Labour Organisation (ILO) classification of radiographs of pneumoconioses was developed to limit variation in classification of parenchymal abnormalities. In this study the manner in which chest radiographs were interpreted in 134 investigations reported in four peer reviewed journals during the five year period 1985-90 was examined. The approach for applying the ILO system was poorly described in most studies. For example, of 86 investigations using more than one reader, 66.3% described the method of reconciliation, but methods were not consistent among investigations. Our results indicate a number of potential problems in application of the ILO system, and gaps in existing recommendations that should be considered.

Experiences of a state-sponsored notification and screening program for asbestos workers

Worker notification can involve a broad range of activities including medical screening, personal and mass communications, cohort identification and tracing, and even litigation. The inclusion or exclusion of various supporting activities in a worker notification program may pose significant medical, public health, financial, logistical, and even legal implications for targeted individuals as well as for the agencies involved. This report describes some experiences in a state-sponsored notification and screening program of approximately 4,500 asbestos workers in Minnesota. In this program, a variety of factors led to the decision to provide medical screening to 1,101 workers and 451 spouses. It is anticipated that another 3,400 workers will be notified but not screened. A follow-up survey of notified workers showed overwhelming support for this program. It is estimated that this program will cost more than $650,000 by its completion. The decision to institute medical screening and other support activities should be made with careful consideration of the diverse implications of these activities to the individuals, communities, and agencies involved.

A comparison of radiographic interpretation of silica exposed workers using the 1963 and the 1986 Chinese roentgenodiagnostic criteria of pneumoconioses

As part of a larger study relating to silica exposure, silicosis, and lung cancer mortality in Chinese mine and factory workers, 1936 old posterior-anterior chest X-rays were re-interpreted according to the 1986 Chinese Roentgenodiagnostic Criteria of pneumoconioses. Each film was independently read by three individuals from a panel of eleven radiologists, and this reading was compared to the original one. Subsequent to the independent readings, a groups of three readers interpreted the films together, called the consensus readings. Comparisons were made by Chinese stage of pneumoconiosis. For the entire cohort, there was a crude agreement of 57.4% between the old and the new interpretations. Agreement within one step of full agreement was 92.5%. The interpretations done by median reading and by consensus were very similar. In general, there was a tendency for the old readings to be slightly higher compared to the new interpretations. This tendency was most marked in the tin mines, followed in decreasing order by the iron/copper mines, the potteries, and the tungsten mines. The agreement between the old and new interpretations is felt to be satisfactory.

Chest radiographs in subjects with asbestos-related abnormalities: comparison between ILO categorizations and clinical reading

The findings of a previous chest X-ray screening, determined without using standardized criteria, were reassessed by means of the ILO classification. Of 470 radiographs that had been determined as showing asbestos-related changes, 430 were categorized according to the ILO Classification. Small opacities with profusion greater than or equal to 1/0 were described in 39 (52%) of 75 participants who, on the original clinical reading, had been determined as having lung fibrosis, and in 45 (12.7%) of 355 who were determined as having pleural changes only. When considering circumscribed pleural thickening at the chest wall or diaphragm, as categorized by the ILO Classification, such changes were present in 401 (93.7%) of 428 subjects with pleural changes as determined on the clinical reading. In addition to the improved sensitivity and specificity achieved, the ILO Classification also allows comparison with other studies. The most apparent disadvantage of the ILO system is that it cannot firmly separate the various types of asbestos-related pleural changes. The study revealed that the previous asbestos exposure of the case subjects had occurred in many different workplaces and occupations.

Variability in interpretation of radiographs for asbestosis abnormalities: problems and solutions

Interobserver variation continues to be a substantial problem for interpretation of asbestos-workers' X-rays, despite incremental improvements in the International Labour Office (ILO) classification system. The apparent degree of agreement/variability is affected by the population disease prevalence. For most populations, experts will be able to agree routinely about normal films. Yet designed trials show that selected experts exhibit a two-fold or greater range for perceived prevalence of abnormal profusions. This degree of variability would be adequate for many population studies were it achieved under real-life conditions of interpretation. Unfortunately, the realistic degree of interobserver variability is probably much higher under actual population surveillance conditions. Asbestos population surveillance programs have provided little or no quality assurance for ILO interpretations, so the very high variability suggested by available data are not surprising and are not the fault of the classification. Recommendations include required participation in quality-assurance panels for those who wish to be certified as interpreters, with consistent feedback and the dropping of outliers. If quality-assurance programs are implemented, the classification will achieve its intended epidemiologic purposes.

A comparison of pneumoconiosis interpretation between Chinese and American readers and classifications

As a preliminary step in joint Sino-American pneumoconiosis research efforts, a formal chest X-ray pneumoconiosis reading trial was conducted among Chinese and American radiologists. Twelve Chinese readers from different institutions located in south central China used the 1986 Chinese Roentgenodiagnostic Criteria of Pneumoconioses. Three American radiologists, centified as NIOSH "B" readers, used the International Labour Office Classification of Pneumoconioses. The chest X-ray study set consisted of 150 posterior-anterior films. One-half were Chinese X-rays of silica-exposed workers, and the other half were American films of variously exposed workers but primarily coal miners. All readings were done independently. The results showed that the inter-reader variability among the Chinese was similar to that of the American readers, both being in an acceptable range. In addition, there was general agreement between the Chinese and American interpretations. For small opacity profusion, the Chinese readers tended to read slightly more diseases than their American counterparts, although there was exact agreement as to the major category in two-thirds of the films. Agreement for film quality, and pleural disease was less, but was not different from reported variation among American "B" readers. Overall, the results suggest that despite the use of two different classification systems, a valid correspondence exists between the Chinese and American Interpretations, which is suitable for use in epidemiologic research.

Radiologic detection of pleural thickening

The purpose of this study was to investigate four aspects of the radiologic detection of pleural thickening: the specificity of chest X-rays read by B readers, the effects of threshold definitions of positivity, the extent of variability among different X-ray readers, and the effect of film quality. A series of 421 chest X-rays corresponding to consecutive autopsies was reviewed by six B readers working independently and using modified International Labour Organization forms. The prevalence of true pleural thickening was approximately 6%. Using a strict definition of positivity, the average specificity was 0.85. Specificity decreased when a more liberal definition was used. (The average sensitivity was 0.32, which appeared to increase with a more liberal definition of positivity; because of small numbers of true positives, the sensitivity estimates were unstable.) Kappa statistics for interreader agreement ranged from 0.06 to 0.40, suggesting considerable interreader variability. Decreasing film quality was associated with decreasing sensitivity, but specificity was not affected by film quality. Because the chest X-ray has suboptimal test characteristics in the diagnosis of pleural thickening and because there is considerable interreader variability, we conclude that caution is required in interpreting radiologic detection of pleural thickening using ILO procedures.