Interactive Exploration of Large Remote Micro-CT Scans Prohaska, Hutanu, Kähler, Hege (Zuse Institut Berlin)

Slides:

Advertisements

Similar presentations

University of Chicago Department of Energy The Parallel and Grid I/O Perspective MPI, MPI-IO, NetCDF, and HDF5 are in common use Multi TB datasets also.

Advertisements

EUFORIA FP7-INFRASTRUCTURES , Grant JRA4 Overview and plans M. Haefele, E. Sonnendrücker Euforia kick-off meeting 22 January 2008 Gothenburg.

Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung Google Jaehyun Han 1.

A Hardware-Assisted Hybrid Rendering Technique for Interactive Volume Visualization Brett Wilson Kwan-Liu Ma University of California, Davis Patrick S.

Interactive Medical Imaging over the Network Centre Hospitalier Universitaire Vaudois Lausanne Raffaele Noro, Olivier Verscheure and Jean-Pierre Hubaux.

1 Asynchronous Bit-stream Compression (ABC) IEEE 2006 ABC Asynchronous Bit-stream Compression Arkadiy Morgenshtein, Avinoam Kolodny, Ran Ginosar Technion.

Distributed Interactive Ray Tracing for Large Volume Visualization Dave DeMarle Steven Parker Mark Hartner Christiaan Gribble Charles Hansen.

The Virtual Microscope Umit V. Catalyurek Department of Biomedical Informatics Division of Data Intensive and Grid Computing.

1/13 A MJPEG Encoder for the NOW to visualize and navigate on data- intensive scenarios José María González

3D Computer Rendering Kevin Ginty Centre for Internet Technologies

Memory Efficient Acceleration Structures and Techniques for CPU-based Volume Raycasting of Large Data S. Grimm, S. Bruckner, A. Kanitsar and E. Gröller.

Fast Isosurface Visualization on a High-Resolution Scalable Display Wall Adam Finkelstein Allison Klein Kai Li Princeton University Sponsors: DOE, Intel,

MATE-EC2: A Middleware for Processing Data with Amazon Web Services Tekin Bicer David Chiu* and Gagan Agrawal Department of Compute Science and Engineering.

Client/Server Software Architectures Yonglei Tao.

Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung Google∗

Report ： Zhen Ming Wu 2008 IEEE 9th Grid Computing Conference.

Parallel HDF5 Introductory Tutorial May 19, 2008 Kent Yang The HDF Group 5/19/20081SCICOMP 14 Tutorial.

Institute of Computer and Communication Network Engineering OFC/NFOEC, 6-10 March 2011, Los Angeles, CA Lessons Learned From Implementing a Path Computation.

What is a Network?. Definition of a computer network A computer network is a system in which computers are connected to share information and resources.

WP-8, ZIB WP-8: Data Handling And Visualization Review Meeting Report Felix Hupfeld, Andrei Hutanu, Andre Merzky, Thorsten Schütt, Brygg Ullmer Zuse-Institute-Berlin.

Test Of Distributed Data Quality Monitoring Of CMS Tracker Dataset H->ZZ->2e2mu with PileUp - 10,000 events ( ~ 50,000 hits for events) The monitoring.

Distributed Visualization and Data Resources Enabling Remote Interaction and Analysis Scott A. Friedman UCLA Institute for Digital.

1 High level view of HDF5 Data structures and library HDF Summit Boeing Seattle September 19, 2006.

HDF5 A new file format & software for high performance scientific data management.

Profiling Grid Data Transfer Protocols and Servers George Kola, Tevfik Kosar and Miron Livny University of Wisconsin-Madison USA.

A Metadata Based Approach For Supporting Subsetting Queries Over Parallel HDF5 Datasets Vignesh Santhanagopalan Graduate Student Department Of CSE.

February 2-3, 2006SRB Workshop, San Diego P eter Cao, NCSA Mike Wan, SDSC Sponsored by NLADR, NFS PACI Project in Support of NCSA-SDSC Collaboration Object-level.

WebCCTV 1 Quadrox WebCCTV Training Technical Quadrox November 2005.

Light-Weight Data Management Solutions for Scientific Datasets Gagan Agrawal, Yu Su Ohio State Jonathan Woodring, LANL.

ICPP 2012 Indexing and Parallel Query Processing Support for Visualizing Climate Datasets Yu Su*, Gagan Agrawal*, Jonathan Woodring † *The Ohio State University.

Swapping to Remote Memory over InfiniBand: An Approach using a High Performance Network Block Device Shuang LiangRanjit NoronhaDhabaleswar K. Panda IEEE.

Types of Operating Systems

Multi-level Raid Multi-level Raid 2 Agenda Background -Definitions -What is it? -Why would anyone want it? Design Issues -Configuration and.

1 HDF5 Life cycle of data Boeing September 19, 2006.

1 Public DAFS Storage for High Performance Computing using MPI-I/O: Design and Experience Arkady Kanevsky & Peter Corbett Network Appliance Vijay Velusamy.

GVis: Grid-enabled Interactive Visualization State Key Laboratory. of CAD&CG Zhejiang University, Hangzhou

Evaluation of a Bricked Volume Layout for a Medical Workstation based on Java Peter Kohlmann, Stefan Bruckner, Armin Kanitsar, M. Eduard Gröller Institute.

SC 2013 SDQuery DSI: Integrating Data Management Support with a Wide Area Data Transfer Protocol Yu Su*, Yi Wang*, Gagan Agrawal*, Rajkumar Kettimuthu.

May 2003National Coastal Data Development Center Brief Introduction Two components Data Exchange Infrastructure (DEI) Spatial Data Model (SDM) Together,

CCGrid, 2012 Supporting User Defined Subsetting and Aggregation over Parallel NetCDF Datasets Yu Su and Gagan Agrawal Department of Computer Science and.

Computer Graphics II University of Illinois at Chicago Volume Rendering Presentation for Computer Graphics II Prof. Andy Johnson By Raj Vikram Singh.

Experiences with OGSA-DAI : Portlet Access and Benchmark Deepti Kodeboyina and Beth Plale Computer Science Dept. Indiana University.

An Update on COLA’s Software Development Jennifer M. Adams and Brian Doty.

SDSC The Scripps Research Institute U. Texas, Austin U.C.L.A. SIO/MIT U. California, Davis Mississippi State U. Scalable Visualization Toolkits for Bays.

Medical Imaging Lection 3.

3/16/04James R. McGirr1 Interactive Rendering of Large Volume Data Sets Written By : Stefan Guthe Michael Wand Julius Gonser Wolfgang Straβer University.

ClinicalSoftwareSolutions Patient focused.Business minded. Slide 1 Opus Server Architecture Fritz Feltner Sept 7, 2007 Director, IT and Systems Integration.

Super Scaling PROOF to very large clusters Maarten Ballintijn, Kris Gulbrandsen, Gunther Roland / MIT Rene Brun, Fons Rademakers / CERN Philippe Canal.

Medical Imaging Lection 3. Basic Questions Imaging in Medical Sciences Transmission Imaging PACS and DICOM.

A Scalable Distributed Datastore for BioImaging R. Cai, J. Curnutt, E. Gomez, G. Kaymaz, T. Kleffel, K. Schubert, J. Tafas {jcurnutt, egomez, keith,

1.3 ON ENHANCING GridFTP AND GPFS PERFORMANCES A. Cavalli, C. Ciocca, L. dell’Agnello, T. Ferrari, D. Gregori, B. Martelli, A. Prosperini, P. Ricci, E.

Interactive Terascale Particle Visualization Ellsworth, Green, Moran (NASA Ames Research Center)

AMGA-Bookkeeping Carmine Cioffi Department of Physics, Oxford University UK Metadata Workshop Oxford, 05 July 2006.

System Optimization Networking

Lecture 01: Computer Architecture overview. Our Goals ● Have a better understanding of computer architecture – Write better (more efficient) programs.

Experiences with VI Communication for Database Storage Yuanyuan Zhou, Angelos Bilas, Suresh Jagannathan, Cezary Dubnicki, Jammes F. Philbin, Kai Li.

Instrument Elements at Work Imperial Brunel Elettra IBM CNIT IASA IMAA GRNet UniUD INFN Monitor GUI Network Information Provider Instrument Element Network.

CIT 140: Introduction to ITSlide #1 CSC 140: Introduction to IT Operating Systems.

Chapter 3 Getting Started. Copyright © 2005 Pearson Addison-Wesley. All rights reserved. Objectives To give an overview of the structure of a contemporary.

Introduction to Distributed Platforms

Introduction to HDF5 Session Five Reading & Writing Raw Data Values

Tamas Szalay, Volker Springel, Gerard Lemson

XROOTd for cloud storage

2018 Huawei H Real Questions Killtest

Comparison of LAN, MAN, WAN

iSCSI-based Virtual Storage System for Mobile Devices

Benchmarking Cloud Serving Systems with YCSB

TeraScale Supernova Initiative

Jeremy Meredith, Lawrence Livermore National Laboratory

Presentation transcript:

Interactive Exploration of Large Remote Micro-CT Scans Prohaska, Hutanu, Kähler, Hege (Zuse Institut Berlin)

Motivation Large medical datasets (10 – 100 GB) Centrally (remotely) stored image data Interactive exploration

Design Overview Goal: efficient external data access Interface to “regular grid data”: –multidimensional array of data elements –subblocks at various resolutions can be accessed. Asynchronous access using a background thread Under the hood: HDF5 as data container

HDF5 and Data Layout HDF5 supports –subblock/ hyperslab selection –external I/O drivers –only synchronous I/O  Data is stored at highest resolution using a “chunked” layout (blocksize 256KB – 2MB) In addition, a low-res “preview version” of data is generated (adds 15% of data)

HDF5 and Remote Data Access When handling remote data, the external I/O driver should know which high-level function it serves (virtual data access layer). GridFTP was chosen as underlying client- server data access architecture.

Rendering Hierarchical multi-resolution approach Spherical ROI visualized in high-res, other data displayed at coarser resolution Octree structure used as multilevel representation of data volume, to find blocks intersecting ROI. 3D texture based volume rendering (octree based back-to-front rendering)

Results Measurements: –Responsiveness (time to load one block) = latency + block size/ bandwidth –Throughput Data server: dual Xeon 1.7GHz with 170GB storage (at 15MB/s read rate) Network latency: 0.2ms (LAN), 360ms (WAN)

Results (Contd.) Dataset: 2048 x 2048 x 1024, 2 bytes per voxel, 8GB of data. Preprocessing (generation of preview version) takes 25 minutes and adds 2GB of extra data. Block sizes used: 128 x 128 x 64 (2MB), 64 x 64 x 32 (256KB)

Actual Results Responsiveness: –2MB block: 1.9s/blk (WAN), 0.7s/blk (LAN) –256K block: 1.4s/blk (WAN), 0.1s/blk (LAN) Throughput: –2MB block: 8.3 MBit/s (WAN), 23MBit/s (LAN) –256K block: 1.4MBit/s (WAN), 20MBit/s (LAN) Basically, as expected. Trade off between responsiveness and throughput.