1. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans Matthias Schubert 1 Joint work with Alexander Cavallaro 2, Franz Graf 1, Hans-Peter Kriegel 1, Marisa Thoma 1 1 Ludwig-Maximilians-Universität München, Database Group 2 Imaging Science Institute, University Hospital Erlangen

2. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 2 Outline ROI Queries on CT-Scans ROI Retrieval Based On a General Height Scale: Simple Solution based on Similarity Search Solution based on Generalized Height Scale kNN Regression for mapping slices Iterative interpolation Experimental Validation Summary

3. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 3 Head => Computer Tomography: CT Scans 23000CWZ8S.0001145710.4.1 1, 16, 31, 46, 61, 76, 91, 106, 121 x y z Height axis 3-dimensional grid of 12 bit grey values Depending on resolution: few MB to multiple GB (Example: 2.25 MB) Image strongly depends on the used scanner and the scan parameters DICOM header contains some meta information for each slice DICOM headers are mostly empty or even misleading <= Feet

4. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 4 Picture Archieving and Communication Systems Scans are stored in Picture Archiving and Communication Systems(PACS) Scan retrieval by patient name, time, DICOM information Querying parts of scans is not supported very well => load and transmit complete scan No access to sub volumes specified by an example

5. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 5 Problems slices outside the ROI increase the transfer volume bottleneck is the LAN:  large transfer times (up to minutes)  bandwidth in LAN is a limiting factor Only transferring the ROI requires tracking it  slice numbers in the target scan are not the same as in the query scan (scan regions vary)  positions might vary between scans and patients (organ positions vary)

6. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 6 Region of interest Query target scan v i in PACS ID of CT scan v i Client Server Client CT scan v q result User-defined 3D ROI Example scan v q + chosen ROI Target scan v i on remote Server Matching ROI in target scan v i Locate ROI Trans- fer ROI Image database (PACS)

7. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 7 Outline ROI Queries on CT-Scans ROI Retrieval Based On a General Height Scale: Simple Solution based on Similarity Search Solution based on Generalized Height Scale kNN Regression for mapping slices Iterative interpolation Experimental Validation Summary

8. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 8 Localization via Similarity Search Example scan v q + chosen ROI Target scan v i on remote Server Matching ROI in target scan v i Locate ROI Trans- fer ROI Short Commings: Requires pre-processing or heavy load on server: For each slice in target scan:  Feature Transformation  Comparison to ROI Feature similarity is influenced by global scan similarity: Scan parameters Patient characteristics => Direct similarity often fails

9. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 9 ROI Query Based on a Generalized Height Scale gen. height scale H CT scan v i in PACS ID of CT scan v i Client Server Client CT scan v q result User-defined 3D ROI Instance-based Regression Height axis (H  scan) Example scan v q + chosen ROI Target scan v i on remote Server Matching ROI in target scan v i Locate ROI Trans- fer ROI H H Iterative Interpolation

10. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 10 Instance-based Regression: scan → H Example scan v q + chosen ROI Target scan v i on remote Server Locate ROI Better: Large training set Provides multiple examples annotated within consensus height space H More stable results Training Database: 2D image features of height-annotated CT slices of multiple scans H H H k-NN query: Consensus height h  H Speed-up Measures: Dimension reduction: RCA Spatial Indexing: X-Tree Bar-Hillel et al: Learning distance functions using equivalence relations, ICML‘03 Berchtold et al: The X-Tree: An index structure for highdimensional data, VLDB‘96 Emrich et al: CT Slice Localization via Instance-Based Regression, SPIE‘10

11. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 11 Iterative Interpolation H → scan Combine Regression Mapping with Interpolation height space H CT scan v i (in PACS) 11 Estimate location of v i in H via regression 1 Interpolate target positions and. for h lb and h ub 2 22 3 3 Verify target positions via regression 3 Refinement Interpolation Accept Result vivi H 0

12. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 12 Outline ROI Queries on CT-Scans ROI Retrieval Based On a General Height Scale: Simple Solution based on Similarity Search Solution based on Generalized Height Scale kNN Regression for mapping slices Iterative interpolation Experimental Validation Summary

13. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 13 Quality of Height Regression (scan → H) Quality and Runtime w.r.t. Training Database size Main Memory Runtimes on original, 175-dimensional Image Features On-disc runtime for dataset of 2103 CT scans (= 0.9 Mio slices) after RCA dimension reduction + X-Tree Indexing: dim 10 => 20 ms With feature generation and dimension reduction:Time / Query= 40 ms Error= 1.98 cm

14. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 14 Validation of ROI Query Pipeline Testing Height Range Queries on 5 manually-annotated Landmarks in 33 CT Scans: lower bound of coccyx lower plate of the 12 th thoracic vertebra sacral promontory lower xiphoid process cranial sternum Volume of origin: 23000BVEFR.0000740833.11.1 Annotation Error (LB + UB): 2.6 cm ROI Query Error (LB + UB): 2.6 – 2.4 cm ROI Query Runtimes: 1.3 – 10 seconds Pays off if 8 slices are saved

15. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 15 Runtime Advantages Retrieval Times for Typical Queries: Test on 20 CT scans of 12,000 slices Complete Retrieval time: 70 s per scan => 70 to 99 % reduction of the retrieved volumes Left kidney 16.8 cm Urinary bladder 9.6 cm Hip to lower L5 4.7 cm Arch of aorta 0.9 cm runtime retrieved slices Runtime [sec] 0 50 10 15 2020 0 % 5 % 10 % 15 % 20 % Retrieved fraction of complete volumes

16. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 16 Outline ROI Queries on CT-Scans ROI Retrieval Based On a General Height Scale: Simple Solution based on Similarity Search Solution based on Generalized Height Scale kNN Regression for mapping slices Iterative interpolation Experimental Validation Summary

17. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 17 Conclusion and Outlook Introduced ROI Query Framework: Great speed-up of CT subvolume retrieval queries Low costs and low error of localization Example-based queries are extensible to queries using anatomical atlases Future Work: Extension of height queries to arbitrary 3D queries Test on alternative, non-medical use cases

18. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 18 Thank you.

19. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 19 Backup: Quality of Height Regression (scan → H) Increased Quality and Runtime with Database size Main Memory RuntimesRCA dimension reduction + X-Tree Runtimes

20. INSTITUTE FOR INFORMATICS DATABASE GROUP Region of Interest Queries in CT Scans 20 Backup: Runtime Advantages Simulated real-wolrd queries of varying heights: For 20 CT scans of 12,000 slices: total retrieval time = 1,400 seconds => 70 to 99 % reduction of the retrieved volumes