1. Feature-aligned 4D Spatiotemporal Image Registration Huanhuan Xu 1, Peizhi Chen 2, Wuyi Yu 1, Amit Sawant 3, S.S. Iyengar 4, Xin Li 1 1 Louisiana State University, USA 2 Xiamen University, China 3 UT Southwestern Medical Center, USA 4 Florida International University, USA International Conference on Pattern Recognition November 14, 2012

2. 2 Problem Definition

3. Related Work 3D pairwise registration + interpolation  4D motion model : depends on the reference domain, only smooth in the spatial domain [Pennec et.al.1999, Marsland et.al.2008, Reinhardt et.al.2008]. 4D image registration : both spatial and temporal smoothness[Bhatia et.al.2004, Metz et.al. 2011]. We propose an improved 3D SIFT feature extraction and matching algorithm, then compute the feature-aligned 4D registration which achieves better landmark predication accuracy. 3

4. Algorithm Framework 4

5. Feature Extraction and Matching Improved 3DSIFT vs. N-SIFT (works on two volume images, N = 3) Better Rotation-invariance Better Scale-invariance Perform improved 3D SIFT between every two consecutive volume images Choose those consistent correspondences that appear in all time frames. 5

6. Comparison results on the lung CT volume image (465x300x20), only show one 2D cross section. 3D feature extraction and matching. Our improved method detects more feature correspondence and has fewer matching error 6 Feature Extraction and Matching

7. 4D Image Registration 7

8. Transformation Model B-Spline local transformation model in 4D to directly incorporate both spatial + temporal smoothness: 8

9. Cost Function 9 Minimize the image intensity changes over time.

10. Geometric Constraint 10

11. Optimization 11

12. Inverse Registration 12

13. Implementation Linear interpolation in the spatial domain for the derivation of intensity values for any point not on a grid. Multi-resolution strategy. 13

14. Experiment Comparison 14

15. Tumor Tracking 15 Segment the tumor in the first frame using a template-guided 3D graph- cut segmentation [Iyengar 2012], then with registration results, we track this tumor in the following second/third time sequence. Bottom depicts the registration of this tumor among different time sequences. Dynamic Tumor Tracking with our registration Max: 0.63 Min: 0 Mean: 0.016 Object: Get tumor motion estimation which facilitate lung tumor radiotherapy planning and management.

16. Limitations Time-consuming –Around 3 hours to get one 4D registration.  Use GPU/high performance computing. Physical intuition –Incorporate biomedical model. Only work on mono-model image –i.e. the illumination condition shouldn’t change 16

17. Acknowledgements Anonymous reviewers Funding Agencies: This work is supported by Louisiana BOR-(RCS)-LEQSF(2009-12)-RD-A-06, NSF-CNS-1205682, CNS-1126739, and National Natural Science Foundation of China No.61170323. Thanks! Project Webpage (papers, slides, & videos): http://www.ece.lsu.edu/xinli/CBiomedicine/TumorTracking.html http://www.ece.lsu.edu/xinli/CBiomedicine/TumorTracking.html My Homepage: http://www.cct.lsu.edu/~huanxu4/ http://www.cct.lsu.edu/~huanxu4/ For questions: Please contact me via hxu4@lsu.eduhxu4@lsu.edu 17