TOPLINE

A study of nearly 1500 US radiologists found high variability in interpretation of low-dose CT lung cancer screens, with false-positive rates ranging from 11% to 22% for baseline screens.

METHODOLOGY

• The American College of Radiology (ACR) developed Lung-RADS with a goal of standardizing interpretation of low-dose CT screens and reducing false-positive rates while maintaining sensitivity for cancer detection. The study looked at variability in radiologists’ interpretation of screens in real-world settings.
• Researchers used data from the ACR Lung Cancer Screening Registry, which distinguishes individual radiologists, linked with Medicare claims files for the years 2015-2021. A total of 1445 radiologists who interpreted ≥ 50 baseline or ≥ 50 post-baseline lung cancer screens were included, with a total of 229,599 screens analyzed.
• The false-positive rate was defined as the percentage of positive screens (Lung-RADS score ≥ 3) in patients without a lung cancer diagnosis within 12 months, while sensitivity was defined as the percentage of positive screens in patients with a lung cancer diagnosis within that timeframe.
• Mixed-effects models were used to analyze variability in radiologists’ false-positive rates, accounting for sampling noise and differences across radiologists in covariates including patient demographics, smoking history, potential exposure to endemic fungal diseases on the basis of state of residence, and Lung-RADS version.

TAKEAWAY

• For baseline screens, the median false-positive rate across radiologists was 15.9% (IQR, 11.3%-21.8%) based on observed data, and 16.3% (IQR, 11.8%-21.8%) based on mixed-effects models. For post-baseline screens, the median false-positive rate was 8.5% (IQR, 5.7%-12.8%) and 8.8% (IQR, 6.3%-11.8%), respectively.
• The average odds ratio between random pairs of radiologists was 2.57 for baseline screens and 2.67 for post-baseline screens — indicating that for any random pair, one radiologist would on average have about twice the odds of calling a false positive as the other.
• Higher false-positive rates were associated with greater sensitivity: Radiologists in the upper two quartiles (aggregate false-positive rate, 24.1%) achieved a sensitivity of 94.4% vs 89.6% in the lower two quartiles (aggregate false-positive rate, 10.9%). That greater sensitivity came at the cost of more than 100 additional false positives per additional true positive.
• Only 1%-2% of the variance in false-positive rates was explained by the included covariates.

IN PRACTICE

“Although Lung-RADS was an important first step in reducing [false-positive rates] relative to what was observed in [the National Lung Screening Trial], this study establishes an ongoing need for further reductions in radiologist-level interpretative variation,” the study authors wrote. They added that interventions widely used in screening mammography, including support tools, structured audits, and AI assistance, could have similar value in lung cancer screening.

SOURCE

The study, led by Paul F. Pinsky, PhD, of the National Cancer Institute in Bethesda, Maryland, was published online in the Journal of the American College of Radiology.

LIMITATIONS

All screens were performed in patients aged 65 years or older enrolled in fee-for-service Medicare, which may introduce selection bias and limit generalizability. Although the data suggest that radiologists are operating on different points on a receiver operating characteristic curve, this is only indirect evidence because the dearth of lung cancer cases per radiologist precluded reliable estimation of individual radiologist curves. Information on radiologists was limited and did not include years in practice or radiology specialty, which could affect interpretation variability.

DISCLOSURES

No specific funding was provided for the study. The authors reported no relevant conflicts of interest.

This article was created using several editorial tools, including AI, as part of the process. Human editors reviewed this content before publication.