The Next Generation McIDAS A Look Towards the Future 21 st IIPS 8.2 San Diego, CA - 11 January 2005 Dave Santek Tom Whittaker Thomas Achtor Space Science & Engineering Center (SSEC), University of Wisconsin - Madison

Outline Basic functionality of McIDAS-X History of McIDAS and current user base The next generation: McIDAS-V –VisAD –IDV (Integrated Data Viewer) Data servers: ADDE and OpenADDE

What is McIDAS ? ( Man computer Interactive Data Access System) –UNIX, PC & Mac capable A synergistic tool that integrates numerous data types into one system First developed in the early 1970s Still in use world-wide at research, operational, educational, and commercial sites Collection of user programs and libraries for visualizing and analyzing geophysical data (focus on environmental satellites)

Key McIDAS Attributes Access to extensive geophysical database Core package plus user-written applications Diverse functionality through software (1 million + lines of code) Extensive visualization capabilities Satellite and NOAAPORT data ingest

McIDAS Functionality Digital Image Processing GIS Applications Weather and Climate Data Analysis and Applications Graphical Displays of Data & Information Gridded Data Processing and Analysis Tools Display & Process Control Utilities

1968 WINDCO generates wind vectors from ATS images 1972 First Generation of McIDAS Runs on Harris /5 with 96 KB of programmable memory and 2-5 MB hard drives 1976 GOES ingest system added to McIDAS 1979 Wichita, OK tornado disaster: Congress directs Operational McIDAS system to be installed at the National Severe Storms Forecast Center 1983 People’s Republic of China funds port of McIDAS software to IBM mainframe 1985 McIDAS runs on PCs under DOS operating system 1987 McIDAS runs on PCs under OS/2 operating system and McIDAS Users’ Group is formed 1991 McIDAS runs on UNIX workstations 1994 Satellite and conventional data is served from UNIX workstations, beginning the use of ADDE (Abstract Data Distribution Environment) 1997 McIDAS Users’ Group sunsets support for mainframe McIDAS

McIDAS Users NOAA – NESDIS, AWC, SPC, TPC, etc. NASA – STS, LaRC, MSFC, JPL Unidata – 130 universities and colleges International – EUMETSAT, ECMWF, Spain, Greece, Mexico, Australia Industry – Boeing, Honeywell, Weathernews, Universal Weather, Meteorlogix, etc.

Why Change? McIDAS software (written in Fortran 77 and C) has a 30 year heritage resulting in limited extensibility potential New visualization concepts cannot be incorporated (4D) Forthcoming environmental satellite data cannot be utilized in the best way (GOES-R & NPOESS operational systems)

The road to the next generation Geostationary Sounders (# of channels) VAS (experimental) GOES Sounder (operational) GIFTS (experimental) (12) (18) (~1600) ABS/HES (operational) time The ABS/HES is the next step from experimental or low spectral resolution to operational high-spectral resolution, high temporal sounders. HIRS (operational) CrIS (operational) IASI (operational) AIRS (operational) Aircraft data (experimental), IMG Ground-based (exp) Pioneering work (theory) IRIS (experimental) (~2400)

Simulated GOES-R winds (left) versus GOES current operational winds (right) GOES R - IHOP simulation 1830z 12 June 02 GOES-8 winds 1655z 12 June 02 Hyperspectral Altitude Resolved Water Vapor Winds

McIDAS-V: Looking to the Future We seek to advance the functionality of McIDAS to meet the challenges of the future while retaining its current capabilities, including user-developed code. We are investigating a means to transition existing McIDAS applications to operate in a new environment that does not have the limitations of the current McIDAS system.

IDV: The Candidate Transition System IDV (Integrated Data Viewer) –Based on SSEC’s VisAD package –Has a data model that is inherently capable of working with data types, both now and in the future –Developed for the education community by Unidata –Designed to replace McIDAS-X and GEMPAK at universities

What is VisAD? Open-source, Java library for building interactive and collaborative visualization and analysis tools Features include: –Powerful mathematical data model that embraces virtually any numerical data set –General display model that supports 2- and 3-D displays, multiple data views, direct manipulation –Adapters for multiple data formats (netCDF, HDF-5, FITS, HDF-EOS, McIDAS, Vis5D, etc.) and access to remote data servers through HTTP, FTP, DODS/OPeNDAP, and ADDE protocols

VisAD’s Data Model Designed to support virtually any numerical data Metadata can be integrated into each data object (units, errors, coordinates, etc.) – carried along through mathematical operations Supports mathematical operations as well as evaluation and resampling of data Supports spatial and temporal co- location of data Supports data sharing among different users, different data sources and different scientific disciplines Gully erosion/deposition from VisAD-based DEMViewer

What is the IDV? Unidata developed scientific analysis and visualization tool Freely available Java™ framework and reference application Provides 2- and 3-D displays of geo-scientific data (plus, of course, animations) Stand-alone or networked application

IDV Features Integrated displays of a variety of data types Support for a variety data access methods Interactive probes User defined formulas and functions Bundling of user preferences Integrated HTML viewer Easy configuration Integrated documentation

Supported Data Formats –Scientific Geo-referenced netCDF data (local or remote) McIDAS/ADDE –Societal GIS - ESRI shapefiles (polygons) –Educational HTML Quicktime NO 2 concentrations over Berlin, Germany – data from netCDF, maps from shapefile The IDV can integrate displays of a variety of geo-scientific and other data including:

Surface observations with Jet Stream winds

EOS AIRS Hyperspectral Data Analysis Applications

Current Efforts at SSEC A mechanism to transform McIDAS graphical displays into the geo-referenced VisAD / IDV display A unified user interface to access image data via ADDE and OPeNDAP The capability to run IDV processes in a background mode A mode to run heritage McIDAS applications OpenADDE: ADDE data servers available as Open Source

What is ADDE? Abstract Data Distribution Environment Key data access method for McIDAS-V Data transfer and reformatting –All data returned in common format: not as stored Sectorizing by geographic coverage and spectral bands TCP/IP remote access through registered port 112 Compressed transmission Variety of clients can access the servers In use for 10 years

What servers are available? McIDAS format: Image, grid, point, text netCDF as grid and point HDF as point MODIS Level 1b and products as images AIRS Level 1b AMSR-E Level 1b Level III NEXRAD AWIPS image GRIB

What clients can be used? McIDAS-X McIDAS-Lite IDV Matlab IDL

What is OpenADDE? Open Source version of the ADDE servers and required McIDAS-X library routines Another method for data providers to serve data –McIDAS sites & non-McIDAS sites –Subsecting Encourage data providers to write servers for new data types Code provided under the GNU Lesser General Public License

What is OpenADDE? Initial release are only the image read servers: –McIDAS Area, MODIS & AIRS HDF, NEXRAD, GINI, AWIPS (?) –McIDAS library routines Navigation and calibration modules, utilities 30% of the McIDAS-X library code Available at –Source code, installation and configuration scripts, sample data

Where’s the data? Servers at NOAA, NASA, SSEC, Unidata Realtime global GOES Realtime global MODIS Level 1b GOES product files Polar composites of satellite data

The Next Generation McIDAS A Look Towards the Future 21 st IIPS 8.2 San Diego, CA - 11 January 2005 Questions about McIDAS-V or OpenADDE? Stop down to the University of Wisconsin booth: 406