1. Center for Subsurface Sensing & Imaging Systems Overview of Image and Data Information Management in CenSSIS David Kaeli Northeastern University Boston, MA kaeli@ece.neu.edu

2. R1 R2 Overview of the Strategic Research Plan Fundamental Science Validating TestBEDs L1 L2 L3 R3 Image and Data Information Management S1 S4 S5 S3 S2 Bio-MedEnviro-Civil

3. R3 Research Thrust Overview  Utilize enabling hardware and software technologies to address CenSSIS barriers  Pursue research in enabling technologies  Develop a common set of tools and techniques to address SSI problems:  Hardware parallelization and acceleration  Software toolboxes  Image database management and tools  Utilize enabling hardware and software technologies to address CenSSIS barriers  Pursue research in enabling technologies  Develop a common set of tools and techniques to address SSI problems:  Hardware parallelization and acceleration  Software toolboxes  Image database management and tools Toolboxes

4. CenSSIS Middleware Tools  Parallelization of MATLAB, C/C++ and Fortran codes using Message Passing Interface (MPI) – a software pathway to exploiting GRID-level resources  Utilizing MPI-2 to address barriers in I/O performance  Building on existing Grid Middleware such as Globus Toolkit, MPICH-G2 and GridPort  Presently illustrating the impact of the GRID on system level projects (tomosynthesis reconstruction)  Parallelization of MATLAB, C/C++ and Fortran codes using Message Passing Interface (MPI) – a software pathway to exploiting GRID-level resources  Utilizing MPI-2 to address barriers in I/O performance  Building on existing Grid Middleware such as Globus Toolkit, MPICH-G2 and GridPort  Presently illustrating the impact of the GRID on system level projects (tomosynthesis reconstruction) MATLAB C/C++ Fortran Parallelization MPI MPICH-G2 UPC

5. Impact on CenSSIS Applications  Reduced the runtime of a single-body Steepest Descent Fast Multipole Method (SDFMM) application by 74% on a 32-node Beowulf cluster Hot-path parallelization Data restructuring  Reduced the runtime of a Monte Carlo scattered light simulation by 98% on a 16-node Silicon Graphics Origin 2000 Matlab-to-C compliation Hot-path parallelization Obtained superlinear speedup of Ellipsoid Algorithm run on a 16-node IBM SP2 Matlab-to-C compliation Hot-path parallelization  Reduced the runtime of a single-body Steepest Descent Fast Multipole Method (SDFMM) application by 74% on a 32-node Beowulf cluster Hot-path parallelization Data restructuring  Reduced the runtime of a Monte Carlo scattered light simulation by 98% on a 16-node Silicon Graphics Origin 2000 Matlab-to-C compliation Hot-path parallelization Obtained superlinear speedup of Ellipsoid Algorithm run on a 16-node IBM SP2 Matlab-to-C compliation Hot-path parallelization Soil Air Mine

6. Tomographic mammography  3D image reconstruction from x-ray projections  Used to detect and diagnose breast cancer  Based on well developed mammography techniques  Exposes tissue structure using multiple projections from different angles  Advantages Accuracy: provides at least as much useful information than x-ray film Flexibility: digital image manipulation, digital storage Structural information: using layered images Safe: low-dose x-ray Lower cost: compared to MRI  3D image reconstruction from x-ray projections  Used to detect and diagnose breast cancer  Based on well developed mammography techniques  Exposes tissue structure using multiple projections from different angles  Advantages Accuracy: provides at least as much useful information than x-ray film Flexibility: digital image manipulation, digital storage Structural information: using layered images Safe: low-dose x-ray Lower cost: compared to MRI

7. Image acquisition/reconstruction process  Acquisition: 11 uniform angular samples along Y-axis  X-ray projection: breast tissue density absorption radiograph  Algorithm: constrained non-linear convergence and iterative process  Uses a Maximum Likelihood Estimation  Acquisition: 11 uniform angular samples along Y-axis  X-ray projection: breast tissue density absorption radiograph  Algorithm: constrained non-linear convergence and iterative process  Uses a Maximum Likelihood Estimation detector X-ray source X Z Y Y Set 3D volume Compute projections Correct 3D volume 3D volume Satisfied ? No Yes Exit Initialization Forward Backward X-ray projections

8. Parallelization approaches  Reduce communication data  Segmentation along Y-axis  Using redundant computation to replace communication  Segmenting along x-ray beam  Reduce communication data  Segmentation along Y-axis  Using redundant computation to replace communication  Segmenting along x-ray beam First approach: Non inter-communication (more computation, less communication) Second approach: Overlap with inter-communication Third approach: Non-overlap with inter-communication (less computation, more communication) exchange data Overlap area

9. Tomosynthesis Acceleration Input data set: phantom 1600x2034x45 Serial implementations runs in 2- 3 hours on a P4 machine Platforms: – SGI Altix system – UIUC NCSA Titan cluster – UIUC NCSA IBM p690 – UMich Hypnos cluster – P4 cluster at MGH Number of processors: 32 Computation: SGI Altix with Itanium 2 processor outperforms the other CPUs Currently moving this work to the GRID and the Pittsburgh Supercomputer Center Prototype running on our GRID system at NU

10. Field Programmable Gate Arrays for Subsurface Imaging  Backprojection for Computed Tomography image reconstruction  Sponsored by Mercury Computer  Finite Difference Time Domain (FDTD) in hardware  Collaboration with Humanitarian Demining project  Retinal Vascular Tracing in real time  Collaboration with Real-time Retinal Imaging project  Phase Unwrapping  Collaboration with 3-D Fusion Microscope project  Diverse problems, similar solutions: FPGAs are particularly well suited for accelerating image processing and image understanding algorithms  Backprojection for Computed Tomography image reconstruction  Sponsored by Mercury Computer  Finite Difference Time Domain (FDTD) in hardware  Collaboration with Humanitarian Demining project  Retinal Vascular Tracing in real time  Collaboration with Real-time Retinal Imaging project  Phase Unwrapping  Collaboration with 3-D Fusion Microscope project  Diverse problems, similar solutions: FPGAs are particularly well suited for accelerating image processing and image understanding algorithms

11. Retinal Vascular Tracing: Register 2-D Image to 3-D in Real Time “Smart Camera” Direction of blood vessel PCI BUS

12. Some Recent Publications on Parallelization “Execution-Driven Simulation of Network Storage Systems,” Y. Wang and D. Kaeli, Proceedings of the 12 th ACM/IEEE International Symposium on Modeling, Analysis of Computer and Telecommunication Systems, October 2004, pp. 604-611. “Profile-guided File Paritioning on Beowulf Clusters,” Y. Wang and D. Kaeli, Journal o f Cluster Computing, Special Issue on Parallel I/O, to appear, “An Object-oriented Parallel Library,” C. Oaurrauri and D. Kaeli, International Journal of High Performance of Computing and Networking, to appear. “Digital Tomosynthesis Mammography using a Parallelized Maximum Likelihood Reconstruction Method,” T. Wu, R. Moore, E. Rafferty, D. Koppans, J. Zhang, W. Meleis and D. Kaeli, Medical Imaging, 5368, February 2004. “Mapping and characterization of applications in Heterogeneous Distributed Systems,” J. Yeckle and W. Rivera, To appear in Proceed. of the 7th World Multiconference on Systemics, Cybernetics and Informatics (SCI2003). “Profile-Guided I/O Partitioning,” Y. Wang and D. Kaeli, Proceedings of the 17 th ACM International Symposium on Supercomputing, June 2003, pp. 252-260. “Source-Level Transformations to Apply I/O Data Partitioning,” Y. Wang and D. Kaeli, Proceedings of the IEEE Workshop on Storage Network Architecture and Parallel IO, Oct. 2003, pp. 12-21. “Execution-Driven Simulation of Network Storage Systems,” Y. Wang and D. Kaeli, Proceedings of the 12 th ACM/IEEE International Symposium on Modeling, Analysis of Computer and Telecommunication Systems, October 2004, pp. 604-611. “Profile-guided File Paritioning on Beowulf Clusters,” Y. Wang and D. Kaeli, Journal o f Cluster Computing, Special Issue on Parallel I/O, to appear, “An Object-oriented Parallel Library,” C. Oaurrauri and D. Kaeli, International Journal of High Performance of Computing and Networking, to appear. “Digital Tomosynthesis Mammography using a Parallelized Maximum Likelihood Reconstruction Method,” T. Wu, R. Moore, E. Rafferty, D. Koppans, J. Zhang, W. Meleis and D. Kaeli, Medical Imaging, 5368, February 2004. “Mapping and characterization of applications in Heterogeneous Distributed Systems,” J. Yeckle and W. Rivera, To appear in Proceed. of the 7th World Multiconference on Systemics, Cybernetics and Informatics (SCI2003). “Profile-Guided I/O Partitioning,” Y. Wang and D. Kaeli, Proceedings of the 17 th ACM International Symposium on Supercomputing, June 2003, pp. 252-260. “Source-Level Transformations to Apply I/O Data Partitioning,” Y. Wang and D. Kaeli, Proceedings of the IEEE Workshop on Storage Network Architecture and Parallel IO, Oct. 2003, pp. 12-21. Held again in 2004

13. CenSSIS Solutionware – UPRM/NU/RPI Toolbox Development  Support the development of CenSSIS Solutionware that demonstrates our “Diverse Problems – Similar Solutions” model  Develop Toolboxes that support research and education  Establish software development and testing standards for CenSSIS Image and Sensor Data Database  Develop an web-accessible image database for CenSSIS that enables efficient searching and querying of images, metadata and image content  Develop image feature tagging capabilities Toolbox Development  Support the development of CenSSIS Solutionware that demonstrates our “Diverse Problems – Similar Solutions” model  Develop Toolboxes that support research and education  Establish software development and testing standards for CenSSIS Image and Sensor Data Database  Develop an web-accessible image database for CenSSIS that enables efficient searching and querying of images, metadata and image content  Develop image feature tagging capabilities Toolboxes

14. Status of the CenSSIS Toolboxes  Hyperspectral Image Analysis Toolbox (HIAT)  October 2004  Multiview Tomography Toolbox (MVT)  fddlib:  January 2003 (v. 1.0)  July 2003 (v. 1.1)  mvt:  October 2004  Rensselaer Generalized Registration Library (RGRL)  September 2004  Hyperspectral Image Analysis Toolbox (HIAT)  October 2004  Multiview Tomography Toolbox (MVT)  fddlib:  January 2003 (v. 1.0)  July 2003 (v. 1.1)  mvt:  October 2004  Rensselaer Generalized Registration Library (RGRL)  September 2004 HIAT MVT RGRL

15. New toolbox: Improving the quality of radiation oncology @ MGH  Developed a 4D (3D + including time) visualization browser tool kit  Visualize Computed Tomography (CT) images, organ outlines (wire contours) and the isodose lines (treatment dosage)  Present all this information in a user friendly interface  Developed a 4D (3D + including time) visualization browser tool kit  Visualize Computed Tomography (CT) images, organ outlines (wire contours) and the isodose lines (treatment dosage)  Present all this information in a user friendly interface

16. 4-D Visualization of Lung Tumors Dosage 4-D Visualization

17. The Future for CenSSIS Toolboxes SCIRun Collaboration with the University of Utah

18.  Deliver an web-accessible database for CenSSIS that enables efficient searching and querying of images, sensor data, metadata and image content  More that 4000 metadata-rich images/datasets presently available online (> 10,000 by 2006)  Database Characteristics: Relational complex queries (Oracle9i) Data security, reliability and layered user privileges Efficient search and query of image content and metadata Content-based image tagging using XML Indexing algorithms (2D, 3D, and 4D) Explore object relational technology to handle collections CenSSIS Image Database System 1 2 3 4 mouse embryo

19. CenSSIS Image Database System

21. Utilize Machine Learning algorithms to improve query view

22. CenSSIS Image Database System Provides data description associated with initial collection, but does not allow for further elaboration or annotation.

23. Image Annotation  Provide the ability to markup image with searchable features  Enable image database to be more effectively data- mined  Provide the ability to markup image with searchable features  Enable image database to be more effectively data- mined Embryo developmental stages 1 cell embryo 2 cell embryo 4 cell embryo

24. XML and Java XML (Extensible Markup Language) Provides maximum flexibility and portability Well-supported standard Powerful querying tools available in Oracle The Java2 Platform Cross-platform compatibility Standard web-browser interface Native XML support

25. Image Tagging A raw image file from the CenSSIS Database QUERY: I want to be able to add to this image textual annotations, providing my medical team with questions about particular ROIs: Difficult to describe regions in an image Difficult to pinpoint specific features in images Global image metadata too coarse to facilitate low-level tagging

26. Image Tagging Image with tags Metadata associated with specific areas Query for specific image features

27. The Image Tagging Interface Drawing Tools View Options Tag Options

28. Tags and XML 101 58 79 46 [custom XML tags go here] awilliam

29. The Future Role of Image Annotation  Provide a vehicle for natural collaboration A richer set of metadata to enable more detailed queries Potential to perform extensive data mining on image content An eye toward content-based image retrieval Tumor tracking paper recently accepted to SIGMOD 2005

30. The CenSSIS Image Database System  Hosts the image and sensor data of CenSSIS (>500 images online)  http://censsis-db1.ece.neu.edu/  Provides metadata indexed image searching  Uses XML tags to allow for easy information interchange  Evolved into a project-based management system, allowing users to organize their data hierarchically  Key issue: how do we develop collaboration tools that increase the value of data stored in the database?  Presently exploring how best to integrate both visualization and image annotation into the existing framework (NIH proposal)  Hosts the image and sensor data of CenSSIS (>500 images online)  http://censsis-db1.ece.neu.edu/  Provides metadata indexed image searching  Uses XML tags to allow for easy information interchange  Evolved into a project-based management system, allowing users to organize their data hierarchically  Key issue: how do we develop collaboration tools that increase the value of data stored in the database?  Presently exploring how best to integrate both visualization and image annotation into the existing framework (NIH proposal)

31. CenSSIS Image and Data Information Management  Addressing key research barriers in computational efficiency, embedded computing and image/sensor data management  Exploiting Grid resources to enable new discovery in SSI applications  Producing a image/data repository and software-engineered Subsurface Sensing and Imaging Toolsets  Developing enabling tools targeting system-level projects Near real-time reconstruction and visualization Visualization of complex motion Predicting motion in image data using database indexing techniques  Addressing key research barriers in computational efficiency, embedded computing and image/sensor data management  Exploiting Grid resources to enable new discovery in SSI applications  Producing a image/data repository and software-engineered Subsurface Sensing and Imaging Toolsets  Developing enabling tools targeting system-level projects Near real-time reconstruction and visualization Visualization of complex motion Predicting motion in image data using database indexing techniques MVT