Pulmonary nodule detection using CAD Alessandra Retico Istituto Nazionale di Fisica Nucleare, Sezione di Pisa, Italy ECR 2013 – March 7-11, Vienna

outline Designing a CAD system Evaluation of stand-alone CAD performance CAD impact on radiologists’ performance Open issues in CAD research A. Retico - INFN Pisa 2

CAD history Computer-aided detection (CAD) of pulmonary lesions dates back to 1963 [Lodwick et al. Radiology (1963)] The scientific literature related to lung CAD has continued to increase since then. A. Retico - INFN Pisa 3 - CAD publications - Lung CAD publications

CAD in lung cancer screening Screening protocols with low-dose CT settings may benefit more of CAD systems for lung nodule detection. CAD used as second reader can improve the radiologists’ detection ability CAD are required to show high sensitivity at low false-positive rate to avoid increasing radiologists’ reading time A. Retico - INFN Pisa 4 U.S. National Lung Screening Trial (NLST) – more then subjects Lung cancer trial results show mortality benefit with low-dose CT Twenty percent fewer lung cancer deaths seen among those who were screened with low-dose spiral CT than with chest X-ray noisy slices per subject to be reviewed Nov

CAD system for lung nodule detection A. Retico - INFN Pisa 5

CAD target: pulmonary nodules A. Retico - INFN Pisa 6 Solid nodules fully embedded in the lung parenchyma: internal nodules Solid nodules attached or connected to the pleura surface: juxta-pleural nodules Part-solid nodules and non-solid nodules (ground glass opacities, GGO)

1. Image preprocessing [Data resampling (homogeneous voxel sizes)] [Noise reduction (Gaussian filters,...)] Lung segmentation Aims at identifying voxels belonging to lungs discarding other anatomical structures and at providing the shape of the pleura surface Main steps: Identification of low intensity voxel inside the patient’s body Trachea segmentation Lung separation Vessels and airways removal [Pu et al. Comput Med Imaging Graph (2008); van Rikxoort et al. Med Phys (2009); De Nunzio et al. J Digit Imaging (2011); Pu et al. IEEE Trans Vis Comput Graph (2011); Masala et al. Computer Physics Communications (2013)] A. Retico - INFN Pisa 7

2. Identification of the nodule candidates Crucial task: nodules missed at this stage can not be recovered in the following steps. Intensity and shape properties of the objects have to be exploited: nodules have relatively higher intensity with respect to lung parenchyma; they are almost rounded in shape. Some approaches: Multiple grey-level thresholding and morphological processing [Armato et al. Med Phys (2001); Bellotti et al. Med Phys (2007); Messay et al. Med Image Anal (2010)] Local adaptive thresholding [Zhao et al. Med Phys (2003); Suárez-Cuenca et al. Comput Biol Med (2009)] k-mean clustering [Ge et al. Med Phys (2005); Sahiner et al. Acad Radiol (2009)] Computation of shape indices [Ye et al. IEEE Trans Biomed Eng (2009), Murphy et al. Medical Image Anal (2009)] Template matching methods [Wang et al. Med Phys (2007); Dehmeshki et al. Comput Med Imaging Graph (2007); Ozekes et al. Korean J Radiol (2008)] A. Retico - INFN Pisa 8

Example: identification of internal nodules A. Retico - INFN Pisa 9 [Li Q, Sone S, Doi K. Med Phys (2003)] Internal nodule [Retico et al. Comput Biol Med (2008); Camarlinghi et al. Nuovo Cimento (2011)] Enhancement of spherical objects and suppression of elongated and planar structures Multi-scale dot-enhancer (MSDE) filter Search for local maxima List of internal nodule candidates 59:293:226:5.0:0:peak1 54:308:213:5.0:0:peak2 175:251:215:5.0:0:peak3 363:249:142:5.0:0:peak4 50:252:243:5.0:0:peak5 323:175:173:5.0:0:peak6 371:150:128:5.0:0:peak7 … FP nodule The list can contain many false positives

nodule A. Retico - INFN Pisa 10 [Paik et al. IEEE Trans Med Imaging (2004)] Juxta-pleural nodule Enhancement of regions with extra curvature trough a gradient-based filter Pleura Surface Normal (PSN) filter Search for local maxima List of juxta-pleural nodule candidates 59:293:226:5.0:0:peak1 54:308:213:5.0:0:peak2 175:251:215:5.0:0:peak3 363:249:142:5.0:0:peak4 50:252:243:5.0:0:peak5 323:175:173:5.0:0:peak6 371:150:128:5.0:0:peak7 … The list can contain many false positives Example: identification of juxta-pleural nodules [Retico et al. Comput Biol Med (2009); Camarlinghi et al. Nuovo Cimento (2011)]

3. False positive (FP) reduction Nodule candidate characterization. The segmentation of the nodule candidate is required A. Retico - INFN Pisa 11 Nodule candidates are generally described in terms of global shape- and/or intensity-based features: Grey-level based features mean intensity, standard deviation, skewness, kurtosis, … Morphological features surface, volume, sphericity, roundness, …

Example: shape/intensity global nodule features A. Retico - INFN Pisa 12 Candidate classification can be performed in sequential steps Cuts on basic features to eliminate obvious false positives More sophisticated classifiers to analyze more complex sets of features The features are classified by e.g. rule-based classifiers, Linear Discriminant Analysis (LDA), k-Nearest-Neighbour (kNN), Artificial Neural Networks (ANNs), Support Vector Machines (SVMs) Each candidate is assigned either the to “nodule” or “healthy tissue” class.

Example: voxel based nodule characterization A. Retico - INFN Pisa 13 normal tissue juxtapleural nodule internal nodule  Voxels classified as nodule  Voxels classified as normal tissue voxel v slice z slice z+1 slice z-1 + other features computed on the voxel neighborhood Each voxel v of a nodule candidate is assigned a vector of features Each vector of features is analyzed by a trained ANN or SVM classifier and be assigned a class membership A majority criterion is adopted to assign nodule candidates to either the “nodule” or “healthy tissue” class [Gori et al. JINST (2007)]

Analysis of CAD FPs A. Retico - INFN Pisa 14 Vessel crossing/branching Pleura thickening Azygos vein

Evaluation of the CAD performance Free Receiver Operating Characteristic curve (FROC) By varying for example the decisional threshold of the CAD classifier different operative points on the FROC curve can be selected A compromise should be reached between the requirement of high sensitivity to lung nodules while avoiding the radiologists are shown too many confounding CAD marks A. Retico - INFN Pisa 15 TP FP

Stand-alone CAD performance Authors Journal and publ. year Dataset characteristicsCAD performance Slice thickness Number of casesNumber of nodules Nodule size (diameter) Sensitivity (%) FP/scan Cascio et al.Comp Biol Med mm 84 CT scans (LIDC)148 nodules >3 mm882.5 Messay et al.Med Image Anal mm 84 CT scans (LIDC)143 nodules (of which about 38 juxta-pleural) 3-30 mm Camarlinghi et al. Nuovo Cimento mm (Italung-CT) 29 CT scans (training); 20 CT scans (validation) 58 nodules (of which 28 juxta- pleural) in training; 38 nodules (of which 15 juxta-pleural) in validation 5-14 mm703 Tan et al.Med Phys mm 125 CT scans (LIDC)80 nodules >3 mm87.54 Golosio et al.Med Phys mm 84 CT scans (LIDC)148 nodules >3 mm794 Murphy et al.Med Image Anal mm > 1500 CT scans (Nelson); 813 CT scans (validation) 1518 nodules in validation set >3 mm804.2 Choi et al.Information Sciences mm 84 CT scans (LIDC) containing 32 CT scans for validation 76 nodules in validation 3-30 mm Sahiner et al.Acad Radiol mm 52 CT scans (LIDC)241 nodules mm545.6 Pu et al.Med Phys mm 52 CT scans184 nodules (of which 58 juxta- pleural, 44 part-solid and 16 non-solid) mm Riccardi et al.Med Phys mm 154 CT scans (LIDC)117 nodules >3 mm716.5 Bellotti et al. Med Phys mm (Italung-CT) 15 CT scans26 (containing 11 juxta-pleural) 5-14 mm Li et al. Acad Radiol 2008 thin slice 117 CT scans153 (solid and GGO) 4-28 mm 86 (81 to GGO) 6.6 Suárez- Cuenca et al. Comp Biol Med mm 22 CT scans77 nodules 4-27 mm807.7 Dehmeshki et al. Comp Med Im Graph mm 70 CT scans178 nodules (containing 20 non- solid or part-solid) 3-20 mm Cascio et al. (2011)Choi et al. (2012) Absolutely unfair: Train, test and validation on different datasets Different validation protocols

LIDC database The National Cancer Institute constituted in 2000 the Lung International Database Consortium (LIDC). Main aim: to develop a large database of annotated spiral CT lung images coming from different centers and manufacturers. A. Retico - INFN Pisa 17 LIDC nodule annotation procedure: No forced consensus between readers: the annotations of 4 radiologists are given. Images reviewed and annotated by four radiologists in double reading phase. Phase1 (Blinded): Each radiologist reads the image and makes his own annotation. Phase2 (Unblinded): Each radiologist reads again the image, with access to the annotations of the other 3 readers. At present consists of 1010 CT scans with annotations

CAD evaluation on common dataset, CAD combination Innovation through competition Example set of 5 CT with public annotations Validation set of 50 CT without public annotations Upload the results and get your FROC curve back Comparison and combination of different CADs system with a common dataset and a common evaluation protocol A. Retico - INFN Pisa 18 [van Ginneken et al. Medical Image Analysis (2010)] [Niemeijer et al. IEEE Trans Med Imaging (2011)] FujitaLab RG-Magic Magic-Ants Pisa team ISICAD Philips FlyerScan

Combination of 3 CAD systems: MAGIC-5 CAD A. Retico - INFN Pisa 19 [Camarlinghi et al. Int J Comput Assist Radiol Surg (2012)] Validation: 69 LIDC cases (validation); nodules annotated by at least 2 radiologists 80% 3 FP/scan

CAD as second reader: observer performance studies Authors Journal and publ. year Number of cases Number of radiologists Nodule size (diameter) Rad. sensitivity (%) Rad. + CAD sensitivity (%) Notes Beigelman- Aubry et al. Am J Roentgenol (2007) 54 pairs of CT scans (screening) 2 ≥ 4 mm57.7, , 69.2ImageChecker CT CAD System, V 2.0, R2 Technology Inc. (now Hologic); comparable reading time Beyer et al.Eur Radiol (2007) 50 CT scans4 ≥ 4 mm6875Siemens LungCAD prototype; longer reading time Fraioli et al. J Thorac Imaging (2007) 200 CT scans (screening) 3 57, 68, 4694, 96, 94 Hirose et al. Acad Radiol (2008) 21 CT scans (with 0.14 FP/scan) 81 (with 0.89 FP/scan) JAFROC study Das et al.Br J Radiol (2008) 77 CT scans mm 80.1, , 84Siemens LungCAD TM White et al. Acad Radiol (2008) 109 CT scans (multicenter) mm Sahiner et al. Acad Radiol (2009) 85 CT scans6 ≥ 3 mm56 (with 0.67 FP/scan) 67 (with 0.78 FP/scan) JAFROC study; improved sensitivity to nodules ≥ 3 mm, not to nodules ≥ 5 mm Bogoni et al. J Digit Imaging (2012) 48 CT scans5≥ 3 mm 4457Commercial CAD, integrated in PACS, FP rate increased only for 2 radiologists; minimal impact on reading time

Open issues in CAD research Optimization of CAD performance: Dedicated algorithms to subtler nodule types, e.g. non-solid nodules [Jacobs et al. Med Image Comput Comput Assist Interv (2011)] Combination of many different approaches to maximize the performance The lesion characterization can add useful information to radiologists A. Retico - INFN Pisa 21 CAD integration in the clinical workflow CAD demonstrated its positive impact on radiologists’ detection ability, but is not widely used, yet CAD have to be accessible, fast, easy to use, have regulatory approval, integrated in the workstation used for image reviewing

Web-based on demand CAD service Web-based on-demand services could allow: to upload a CT exam; to be notified when CAD output is available for downloading. Huge image processing tasks are remotely executed by high-power processing nodes. Cloud computing is accessible via secure web protocols. Patients’ data are securely handled A. Retico - INFN Pisa 22 [Berzano et al., IEEE NSS-MIC CR, 2012] WIDEN (Web-based Image and Diagnosis Exchange Network) handles the workflow, the image upload and the CAD result notification.

Conclusion Many CAD systems show competitive performance on large screening CT datasets There is still room for improvement:  Detection of subtler nodule types  Enhancing CAD performances by combining different approaches Observer performance studies demonstrated that the use of CAD improves the radiologists’ detection performance CAD should be more effectively integrated in the radiologists’ workflow to save reading time. A. Retico - INFN Pisa 23

Acknowledgments Thank you for your kind attention! Istituto Nazionale di Fisica Nucleare (INFN) supported research on CAD development within the MAGIC-5 and M5L projects (CSN5, ). Thanks are due to all the researchers of the MAGIC-5 and M5L INFN Collaborations. The MAGIC-5 CAD systems have been developed and validated in collaboration with: Dr. A. De Liperi and Dr. F. Falaschi (U.O. Radiodiagnostica 2, Azienda Ospedaliera Universitaria Pisana, Pisa, Italy); Prof. D. Caramella, Dr. M. Barattini, Dr. R. Scandiffio (Ricerca Traslazionale e delle Nuove Tecnologie in Medicina e Chirurgia, University of Pisa, Italy); Dr. M. Torsello and Dr. I. Zecca (U.O. Radiologia, Presidio Ospedaliero V. Fazzi, ASL, Lecce). A. Retico - INFN Pisa 24