1. Montage: Experiences in Astronomical Image Mosaicking on the TeraGrid http://montage.ipac.caltech.edu/ ESTO

2. Contributors Nathaniel Anagnostou IPAC Bruce Berriman IPAC Ewa Deelman ISI John Good IPAC Joseph C. Jacob JPL Daniel S. Katz JPL Carl Kesselman ISI Anastasia Laity IPAC Thomas Prince JPL Gurmeet Singh ISI Mei-Hui Su ISI Roy Williams CACR IPAC Infrared Processing and Analysis Center, JPL/Caltech ISI Information Science Institute, USC JPL Jet Propulsion Laboratory CACR Center for Advanced Computing Research, Caltech

3. Large Astronomical Image Archives IRAS1 GB1 arcminAll Sky4 Infrared Bands DPOSS4 TB1 arcsecAll Northern Sky1 Near-IR, 2 Visible Bands 2MASS10 TB1 arcsecAll Sky3 Near-Infrared Bands SDSS-DR25 TB0.4 arcsec 3,324 square degrees (16% of Northern Sky) 1 Near-IR, 4 Visible Bands IRAS2MASS DPOSS SDSS IRAS SDSS-DR2 DPOSS 2MASS Total Image Size Comparison Wavelength Comparison

4. Science Drivers for Mosaics Many important astrophysics questions involve studying regions that are at least a few degrees across Need high, uniform spatial resolution BUT… cameras give high resolution or wide area but not both => need mosaics Mosaics can reveal new structures & open new lines of research Star formation regions, clusters of galaxies must be studied on much larger scales to reveal structure and dynamics Mosaicking multiple surveys to the same grid – image federation – required to effectively search for faint, unusual objects, transients, or unknown objects with unusual spectrum

5. Montage at a High Level Delivers custom, science grade image mosaics User specifies projection, coordinates, spatial sampling, mosaic size, image rotation Preserves astrometry & flux Designed with modular “toolbox” design Loosely-coupled engines for Image Reprojection, Background Rectification, Co-addition Control testing and maintenance costs Flexibility; e.g., custom background algorithm; use as a reprojection and co-registration engine Implemented in ANSI C for portability Public service will be deployed on the TeraGrid Mosaics will be ordered through web portal

6. Example Mosaics 2MASS 3-color mosaic of galactic plane 44 x 8 degrees; 36.5 GB per band; CAR projection 158,400 x 28,800 pixels; covers 0.8% of the sky 4 hours wall clock time on cluster of 4 x 1.4-GHz Linux boxes 100 µm sky; aggregation of COBE and IRAS maps (Schlegel, Finkbeiner and Davis, 1998) 360 x 180 degrees; CAR projection

7. Montage Users Montage Adopted by Spitzer Legacy Teams SWIRE uses Montage for: Building sky simulations for use in mission planning Fast background rectification and co-addition of in-flight images GLIMPSE uses Montage for: Large scale infrared mosaics of the Galactic Plane Quality Assurance of GLIMPSE Spitzer data by co-registration of 2MASS J, H, K and MSX 8  m images Color composite of co-registered 2MASS and MSX. Each square is 0.5 x 0.5 degrees Right: Spitzer IRAC 3 channel mosaic (3.6  m in green, 4.5  m in red, and i-band optical in blue); high redshift non-stellar objects are visible in the full resolution view (yellow box).

8. Montage Users (cont.) Spitzer Space Telescope Outreach IRSA (NASA’s InfraRed Science Archive) COSMOS (a Hubble Treasury Program) Michelson Science Center Navigator Program ground-based archives NSF National Virtual Observatory (NVO) Atlasmaker project UK Astrogrid project

9. Public Release of Montage Source code and user’s guide available for download at http://montage.ipac.caltech.edu/ First release: Montage version 1.7 Emphasized accuracy in photometry and astrometry Testing on synthetic images showed flux errors of < 0.1% and location errors of < 0.1 pixel Images processed serially Extensively tested (2,595 test cases) and validated on 2MASS 2IDR images for 10 WCS projections with mosaics smaller than 2 x 2 degrees and coordinate transformations Equ J2000 to Galactic and Ecliptic Second release: Montage version 2.2 More efficient reprojection algorithm: up to 30x speedup Improved memory efficiency: capable of building larger mosaics Enabled for parallel computation with MPI Enabled for processing on TeraGrid using standard grid tools

10. Montage v1.7 Reprojection: mProject module Arbitrary Input Image Central to the algorithm is accurate calculation of the area of spherical polygon intersection between two pixels (assumes great circle segments are adequate between pixel vertices) Input pixels projected on celestial sphere Output pixels projected on celestial sphere SIMPLE = T / BITPIX= -64 / NAXIS = 2 / NAXIS1= 3000 / NAXIS2= 3000 / CDELT1= - 3.333333E-4 / CDELT2= - 3.333333E-4 / CRPIX1= 1500.5 / CRPIX2= 1500.5 / CTYPE1=‘RA---TAN’ CTYPE2=‘DEC--TAN’ CRVAL1= 265.91334 / CRVAL2= -29.35778 / CROTA2= 0. / END FITS header defines output projection Reprojected Image

11. Transform directly from input pixels to output pixels Approach developed by Spitzer for tangent plane projections Performance improvement in reprojection by x 30 Montage v2.2 Reprojection: mProjectPP module Montage version 2.2 includes a module, mTANHdr, to compute “distorted” gnomonic projections to make this approach more general Allows the Spitzer algorithm to be used for other projections (in certain cases) For “typical” size images, pixel locations distorted by small distance relative to image projection plane Not applicable to wide area regions

12. 1 2 3 mProject 1mProject 2mProject 3 1 2 3 mDiff 1 2mDiff 2 3 mFitplane D 12 mFitplane D 23 ax + by + c = 0dx + ey + f = 0 a 1 x + b 1 y + c 1 = 0 a 2 x + b 2 y + c 2 = 0 a 3 x + b 3 y + c 3 = 0 mBackground 1mBackground 2mBackground 3 1 2 3 mAdd Final Mosaic D 12 D 23 Montage Workflow mConcatFit mBgModel ax + by + c = 0 dx + ey + f = 0

13. Montage Parallelization with MPI 1 2 3 mProject 1mProject 2mProject 3 1 2 3 mDiff 1 2mDiff 2 3 mFitplane D 12 mFitplane D 23 ax + by + c = 0dx + ey + f = 0 a 1 x + b 1 y + c 1 = 0 a 2 x + b 2 y + c 2 = 0 a 3 x + b 3 y + c 3 = 0 mBackground 1mBackground 2mBackground 3 1 2 3 mAdd Final Mosaic D 12 D 23 mConcatFit mBgModel ax + by + c = 0 dx + ey + f = 0 mProject and mBackground jobs parallelized with MPI Round-robin by input image

14. Montage Parallelization with MPI 1 2 3 mProject 1mProject 2mProject 3 1 2 3 mDiff 1 2mDiff 2 3 mFitplane D 12 mFitplane D 23 ax + by + c = 0dx + ey + f = 0 a 1 x + b 1 y + c 1 = 0 a 2 x + b 2 y + c 2 = 0 a 3 x + b 3 y + c 3 = 0 mBackground 1mBackground 2mBackground 3 1 2 3 mAdd Final Mosaic D 12 D 23 mConcatFit mBgModel ax + by + c = 0 dx + ey + f = 0 mDiff and mFitplane jobs parallelized with MPI Round-robin by input image pair

15. Montage Parallelization with MPI 1 2 3 mProject 1mProject 2mProject 3 1 2 3 mDiff 1 2mDiff 2 3 mFitplane D 12 mFitplane D 23 ax + by + c = 0dx + ey + f = 0 a 1 x + b 1 y + c 1 = 0 a 2 x + b 2 y + c 2 = 0 a 3 x + b 3 y + c 3 = 0 mBackground 1mBackground 2mBackground 3 1 2 3 mAdd Final Mosaic D 12 D 23 mConcatFit mBgModel ax + by + c = 0 dx + ey + f = 0 mAdd job parallelized with MPI By block of rows in output mosaic

16. Montage Parallel Performance 6 x 6 degrees 2MASS mosaic of M16 at 1” sampling Time (Days)Number of Nodes Algorithm1248163264128 mProject2026.91042.9540.4279.3150.386.258.441.4 mProjectPP223.5119.574.152.335.029.729.325.8 So, how long would it take to mosaic the entire sky?

17. Montage on the Grid “Grid” is an abstraction Array of processors, grid of clusters, … Use a methodology for running on any “grid environment” Exploit Montage’s modular design in an approach applicable to any grid environment Describe flow of data and processing (in a Directed Acyclic Graph - DAG), including: Which data are needed by which part of the job What is to be run and when Use standard grid tools to exploit the parallelization inherent in the Montage design Build an architecture for ordering a mosaic through a web portal Request can be processed on a grid Our prototype uses the Distributed Terascale Facility (TeraGrid) This is just one example of how Montage could run on a grid

18. Montage on the Grid Using Pegasus (Planning for Execution on Grids) Example DAG for 10 input files mAdd mBackground mBgModel mProject mDiff mFitPlane mConcatFit Data Stage-in nodes Montage compute nodes Data stage-out nodes Registration nodes Pegasus Grid Information Systems Information about available resources, data location Grid Condor DAGMan Maps an abstract workflow to an executable form Executes the workflow MyProxy User’s grid credentials http://pegasus.isi.edu/

19. Montage TeraGrid Portal Abstract Workflow mGridExec Pegasus Concrete Workflow Condor DAGman Grid Scheduling and Execution Service ISI Abstract Workflow Image List DAGMan TeraGrid Clusters SDSC NCSA ISI Condor Pool Computational Grid Location, Size, Band User Portal JPL mDAGFiles Abstract Workflow Service JPL m2MASSList 2MASS Image List Service IPAC mNotify User Notification Service IPAC

20. Montage w/ Pegasus Performance 6 x 6 degree 2MASS Mosaic, computed on NCSA TeraGrid using Pegasus Portal and Montage version 3.0 beta 6

21. Montage on the Grid Performance MPI version: Best performance Requires a set of processors with a shared file system Pegasus/DAGman version Very good performance Built-in fault tolerance Can use multiple sets of processors

22. Summary Montage is a custom astronomical image mosaicking service that emphasizes astrometric and photometric accuracy Second public release, Montage version 2.2, available for download at the Montage website A prototype Montage service has been deployed on the TeraGrid It ties together distributed services at JPL, Caltech IPAC, and ISI Montage at SC2004: See ISI at ANL booth for demo of Pegasus portal running Montage on the TeraGrid See us at NASA and Caltech CACR booths to hear this presentation again or for general questions about Montage Montage website: http://montage.ipac.caltech.edu/