SMART Model Matches Radiologists in Differentiating New Lung Malignancies

The Diagnostic Dilemma of the New Pulmonary Nodule

Clinicians frequently encounter indeterminate lung nodules in patients with a history of malignancy, a clinical challenge exacerbated by an aging population facing an increasing total number of cancer deaths [1]. These lesions may represent a metastatic recurrence or a second primary malignancy, yet established guidelines for other conditions, such as thyroid nodules or hepatocellular carcinoma, often lack specific protocols for differentiating primary from metastatic disease in the post-oncology setting [2, 3]. Current diagnostic pathways rely heavily on qualitative imaging interpretation, but intratumor heterogeneity (the existence of multiple, genetically distinct cell subpopulations within different regions of a single tumor) can make visual differentiation difficult [4, 5, 6]. Because this biological diversity can result in varied imaging features across a single mass, a multicenter study now evaluates a standardized tool to assist radiologists in identifying specific imaging biomarkers. This effort addresses the critical need for improved reproducibility (the ability to yield the same diagnostic result in the same patient across different imaging systems), which is essential for accurate clinical staging and treatment planning [7].

Identifying Key Semantic Imaging Biomarkers

To determine whether specific visual patterns can reliably distinguish new primary tumors from metastatic spread, researchers analyzed data from the multicenter retrospective AI-SONAR biomarker study (IRAS 331656 REC 23/NE/0151). The team evaluated 651 single-timepoint, pre-treatment computed tomography (CT) thorax scans. Of these, 649 scans were technically usable for the final analysis, comprising 299 cases of second primary lung cancer and 350 cases of lung metastasis. To establish a baseline for visual interpretation, nine thoracic oncology radiologists reviewed the scans to evaluate eight semantic features (radiologist-defined imaging characteristics used to describe the qualitative appearance of a lesion, such as its shape or border). The study then employed logistic regression analysis (a statistical method used to determine the strength of the relationship between specific variables and a binary outcome) to identify which of these features significantly differentiated a second primary malignancy from metastatic disease.

The analysis identified several imaging biomarkers that strongly correlated with the etiology of the new malignancy, providing clinicians with concrete visual cues to guide diagnosis. Emphysema was significantly associated with second primary lung cancer (p < 0.0001, odds ratio 0.20 [95% CI 0.14 to 0.29]), as was the presence of an irregular contour (p < 0.0001, odds ratio 0.31 [95% CI 0.20 to 0.48]). Additionally, spiculation (needle-like projections extending from the margin of a pulmonary nodule) served as a significant indicator for a second primary tumor (p = 0.013, odds ratio 0.51 [95% CI 0.30 to 0.89]). In this statistical context, an odds ratio of less than 1 indicates a stronger association with a primary lung malignancy. Conversely, the researchers found that peripheral lung distribution was more common in lung metastasis (p = 0.003, odds ratio 1.80 [95% CI 1.20 to 2.68]), where an odds ratio greater than 1 signifies a higher likelihood of metastatic spread.

SMART Model Performance vs. Clinical Interpretation

Building on these visual biomarkers, the researchers developed the Second Malignancy Aetiology Recognition Tool (SMART) model to help automate the differentiation between second primary lung cancer and metastatic disease. Derived from the previously identified semantic imaging features, this model achieved an Area Under the Curve (AUC) of 0.81 (95% CI 0.78 to 0.84). The AUC is a standard metric representing a model's ability to correctly distinguish between two clinical groups, where a value of 1.0 indicates perfect discrimination. In the primary analysis of all 649 technically usable scans, the SMART model achieved a lung metastasis classification accuracy of 75%.

To evaluate the clinical utility of the tool, the researchers compared the SMART model performance to real-world clinical reader performance using McNemar's test (a statistical method used to compare the accuracy of two classification methods applied to the same dataset). While the model reached 75% accuracy, the radiology readers achieved a lung metastasis classification accuracy of 69%. This difference in diagnostic accuracy between the SMART model and the human readers was statistically significant, yielding a McNemar p-value of less than 0.01.

The study further examined a subset of 550 out of 649 cases where the radiologists provided a specific, confident prediction of either second primary lung cancer or lung metastasis. In this specific cohort, the SMART model accuracy for lung metastasis was 74%, while the reader accuracy for lung metastasis was 77%. Unlike the broader analysis, the difference in accuracy for this 550-case subset was not statistically significant, as indicated by a McNemar p-value of 0.20. These findings suggest that while the SMART model provides a more consistent baseline for classification across all indeterminate cases, its performance is highly comparable to expert thoracic radiologists when those experts feel confident enough to make a definitive diagnosis.

Clinical Implications for Patient Stratification

The clinical utility of the SMART model lies in its ability to provide a standardized assessment in a diagnostic area prone to human variability. In this study of 649 technically usable scans, which included 299 cases of second primary lung cancer and 350 cases of lung metastasis, the researchers observed that radiologist readers called second primary lung cancer more often than lung metastasis in new lesions following prior treated cancer. This tendency toward diagnosing a new primary malignancy can significantly alter the clinical pathway. The management of a second primary tumor often involves curative-intent surgical resection or radical radiotherapy, whereas metastatic disease typically dictates a shift toward systemic therapy or palliative interventions.

The data demonstrate that the SMART model performed comparably with expert thoracic radiologists in the diagnosis of lung metastasis, offering a potential safeguard against diagnostic error. While the model achieved a lung metastasis classification accuracy of 75% compared to 69% for readers across the total 649 cases (McNemar p-value < 0.01), the performance was even more closely matched in the subset of 550 cases where radiologists provided a definitive prediction. In those instances, the SMART model lung metastasis classification accuracy was 74% versus 77% by the reader, a difference that did not reach statistical significance (McNemar p-value 0.20). These results suggest that the model is particularly useful in providing a consistent baseline for classification, especially when imaging features are highly indeterminate.

For the practicing clinician, the integration of semantic imaging features into a formal diagnostic model could improve the stratification of malignant new lung lesions after prior cancer. By refining the differentiation between a second primary malignancy and a metastatic recurrence, tools like the SMART model may lead to earlier, more accurate diagnoses and more precise treatment selection. Optimizing this initial diagnostic step is essential for improving downstream patient outcomes, ensuring that therapeutic resources are directed appropriately based on the specific biology of the new pulmonary lesion.

References

1. Jemal A, Thomas A, Murray TS, Thun MJ. Cancer Statistics, 2002. CA A Cancer Journal for Clinicians. 2002. doi:10.3322/canjclin.52.1.23

2. Haugen BR, Alexander EK, Bible KC, et al. 2015 American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid. 2015. doi:10.1089/thy.2015.0020

3. Heimbach JK, Kulik L, Finn RS, et al. AASLD guidelines for the treatment of hepatocellular carcinoma. Hepatology. 2017. doi:10.1002/hep.29086

4. Gerlinger M, Rowan AJ, Horswell S, et al. Intratumor Heterogeneity and Branched Evolution Revealed by Multiregion Sequencing. New England Journal of Medicine. 2012. doi:10.1056/nejmoa1113205

5. Network TCGA. Comprehensive molecular portraits of human breast tumours. Nature. 2012. doi:10.1038/nature11412

6. Network TCGA. Comprehensive molecular characterization of human colon and rectal cancer. Nature. 2012. doi:10.1038/nature11252

7. Boellaard R, Bolton RCD, Oyen WJ, et al. FDG PET/CT: EANM procedure guidelines for tumour imaging: version 2.0. European Journal of Nuclear Medicine and Molecular Imaging. 2014. doi:10.1007/s00259-014-2961-x