Presentation is loading. Please wait.

Presentation is loading. Please wait.

HPC Advances in LIDAR E2E Processing

Similar presentations


Presentation on theme: "HPC Advances in LIDAR E2E Processing"— Presentation transcript:

1 HPC Advances in LIDAR E2E Processing
UNCLASSIFIED HPC Advances in LIDAR E2E Processing Anthony Galassi NGA/InnoVision September 21, 2011 2011 HPC Challenge Introduction Welcome attendee, introduce the briefing, explain what to expect. Use an opening story, tailored to the audience, to demonstrate the impact of GEOINT. TRANSITION: That’s just one of hundreds of examples that demonstrate the real and significant impact GEOINT is making every day to save lives and secure our nation. UNCLASSIFIED Approved for Public Release – Case #

2 HPC Processing SOA in the Cloud Hybrid Cloud Approach Deployed Concept
Agenda HPC Processing SOA in the Cloud Hybrid Cloud Approach Deployed Concept SOA: Service Oriented Architecture

3 National Geospatial-Intelligence Agency (NGA)
NGA Mission: To provide timely, relevant, and accurate GEOINT in support of national security NGA is both a national intelligence agency and a combat support agency. We get our money, our people, and our guidance from both the Intelligence Community and the Department of Defense. NGA’s mission is to produce timely, relevant, and accurate geospatial intelligence – or GEOINT – in support of national security objectives. NGA has a long, rich history that includes the tradecrafts of imagery and mapping. Formerly known as the National Imagery and Mapping Agency – or NIMA. Name changed to the National Geospatial-Intelligence Agency on Nov 24, This change was part of the 2004 Defense Authorization Bill. The change was simply the latest step in a transformation process, under way since our inception, to introduce the new intelligence discipline of geospatial intelligence within the Intelligence Community. I will talk more about what GEOINT is and why it is important as we move through the brief. * NGA is the lead federal agency responsible for Geospatial Intelligence (GEOINT) Approved for Public Release – Case #

4 HPC Processing Implementation
Multi-Source Sensor Data Multi-Source Sensor Data Multi-Source Sensor Data Products created by HPC are used by scientists, analysts and warfighter. Extraction Manipulation Orientation Correction De-noising Calibration Fusion Registration Output

5 LIDAR CMMD Implementation
Metadata and Wideband data are carried together throughout the LIDAR Processing Chain Sensor Data Metadata HDF5/XML/KLV file Necessary metadata (profiles) are determined by analyzing products created by LIDAR scientists, exploitation/tool requirements, and V&V activities LIDAR Processing Chain ALIRT HALOE Bathymetric Other LIDAR Data Extraction Tiling Swath Rotation De-noising PSF Calibration Flattening Registration Geolocation ALIRT: Airborne Ladar Imaging Research Test bed HALOE: High Altitude LIDAR Operations Experiment CMMD: Conceptual Model and Metadata Dictionary  Standard PSF: Point Spread Function HDF-5: Hierarchical Data Format KLV: Key Length Variable XML: eXtended Markup Language V&V: Verification and Validation LIDAR Data Model and dictionary encompass the processing chain LIDAR Data Model LIDAR Data Dictionary End-to-End Processing

6 Processor Tasking, Scripts
SOA on HPC Raw Sortie Data Mass Storage Devices File Copy Data HPC Logs and Results SOA Processor Tasking, Scripts User Input and Review Workflow Scripts: Discovery Status Job Requests Workflow Scripts Processor Tasking HPC Processing is orchestrated by SOA Technology

7 SOA Processing of Large Data Sets
Shows 6 stage LIDAR processing chain with the ability to set parameters for each stage. Can run large number of processing nodes through this single interface. Allows for an unlimited number of product types to be built by this single interface. Custom Products are made using the SCP interface

8 Hybrid Cloud Approach Heavy Iron & Cloud Components
Classified Time Dominant (In Theater) Data delivered to processing environments via transportable disk-drives Government Networks (Sensitive Data) Products are made available to the warfighter through web services Processing of Terabytes of data occurs in theater and is available remotely via web services/cloud Data is processed and accessed by analysts in CONUS through web services and analyst tools in the “cloud” Config Small (<1 MW) Med (1-4 MW) Large (>4 MW) Cores 19, , ,440 Storage PB PB PB Further Processing of Gigabytes of data occurs at CONUS Cloud Computing facility and results are made available remotely via web services/cloud Government Networks (Sensitive Data) Classified Long-term (Mapping)

9 NGA/IID Leads the Migration of HPC Technology
NGA/IB NGA/S Sandia Algorithms Feedback Sensor Data (HALOE, ALIRT) Algorithm Testing Automated and Custom Product Design Prototypes, Demos Prototype Processing Hosting on External HPC, Clouds Optimization on SNL Super Computer Migration to Operations LIDAR Product Evaluations Initial CWG, LIFG Support HPC Hosting (Dark Storm) Theater COMMS Connection NGA/P NGA/ ACQ/OCIO Other Locations

10 Amazon Cloud Experiment Proved Remote Access
Using dedicated hardware solutions for High Performance Computing (HPC) is very expensive and time consuming. The cloud approach allows computing resources to be shared among numerous programs, allows dormant or unused resource to be allocated to immediate needs, and allows more computing resources to be added as needed. The raw (L1) data is too large to move and is not readily available to analysts/analyst tools. Provide access for the analyst (and tools) to the L2 data and the product to be analyzed. Don’t waste time moving raw data around. For unclassified research, commercial vendors such as Amazon can be used to build on-demand HPC clusters and analyst workstations. For sensitive data, NGA Cloud environments can be developed/leveraged. Data from Sortie 33, Partition 25, unclass collection over Wash. D.C. Data collect was over DC. This partition was approx km in length L0  L1 size increase due to translation of sensor data into 3-D data points with more information. Level Data Type Typical Size L0 Raw Data 41GB L1 3D Point Clouds 460 GB (10-12x) L2 De-noised 3D Point Clouds 96GB (down 4.5x) L3 Geo-registered L2 3D Point Clouds ~96GB (1x) L4 DEMs, viewsheds or other standard products 10-20 MB (down 500x) L5 Specialized Products 10-20 MB (1x)

11 HPC-based LIDAR Processing in the National Interest
The Next 15 years HPC-based LIDAR Processing in the National Interest GPUs Faster Access Storage Distributed Processing for extremely large data sets Cloud Computing 1. GPUs - We are currently working with OpenCL and Cuda development environments to build libraries and processing modules that will process large LIDAR data sets. Our priority has been to use the OpenCL interfaces so that our code will be platform independent. We expect GPU and CPU-based processing to converge in the coming years, so we want to be sure that our architecture can use the new components. 2. Faster Access Storage - We have included SSD storage in our current planned architecture, and we are closely following this rapidly moving technology area. Given the incredible speeds possible with SSD Flash memory, and the even more amazing Phase Change memory on the horizon, our platform designs leverage this capability by keeping as much data in RAM and shared memory as possible. 3. Distributed Processing for extremely large data sets - In the current architectures, data storage tends to be centralized, in that individual sensor collections are put in a single file. While this approach is convenient for the storage/archive folks, it is the least efficient method from the data processing perspective. In the coming years, we will be looking to distribute data and processing across the enterprise, which will help address this bottleneck. 4. Cloud Computing - We are currently focusing on the Amazon Cloud (EC2) cloud approach for provisioning processing through Virtual Computers, and storage. However, due to the limitations of the surrounding network, most of our work focuses on solutions for a single supercomputer or data center. As improvements in the way that data is stored occur, we expect to take greater advantage of the cloud. Much of the data the NGA works with is unclassified, so we are hopeful that the agency will find ways to take greater advantage of commercial resources at places like Amazon, RackSpace and Cloud.com.

12 C/Advanced Data Interoperability NGA/IID
For More Information Anthony Galassi C/Advanced Data Interoperability NGA/IID

13 Approved for Public Release – Case #


Download ppt "HPC Advances in LIDAR E2E Processing"

Similar presentations


Ads by Google