QuakeSim Work: Web Services, Portlets, Real Time Data Services Marlon Pierce Contributions: Ahmet Sayar, Galip Aydin, Mehmet Aktas, Harshawardhan Gadgil, Zhigang Qi, Zao Liu Community Grids Lab Indiana University Project Funding: NASA AIST, ACCESS

QuakeSim Project Overview QuakeSim has been funded by CT, AIST, and ACCESS Collaboration of geophysicists and computer scientists to build cyber-infrastructure for geophysical research. CI research and development includes –Portlet-based portals AJAX enabled –Geographical Information System services –Application services to run codes.

QuakeSim Project Development Overview Portlet-based portal components allow different portlets to be exchanged between projects. –Form-based portlets --> Interactive Maps –These are clients to Web services –Share with collaborators of REASoN portal. Sensor Grid: Topic based publish-subscribe systems support operations on streaming data. Web services allow request/response style access to data and codes. –GIS services (WMS, WFS) –“Execution grid” services for running codes: RDAHMM, ST_Filter Use application specific WSDL on top of generic code management services. –GPS daily archive Web Services provided by Scripps.

Evolving Project Philosophy Methodology for interoperability between portal projects exists today. –Portlets, web services –Scripps and IU are proving this. Must continue to move away from the portal priesthood and towards a “scientific mashup” model. –We still need to develop interesting services and client libraries. –But we should make it easy for the application scientists to Add their own applications. Add their own data. Make their own web applications (ie mashups)

IU Portlet Development We use JSR 168 portlets to build sharable portal plugins.

Portlet Summary RDAHMMSet up and run RDAHMM, query Scripps GRWS GPS Service, maintain persistent user sessions. ST_FilterSimilar to RDAHMM portlet; ST_Filter has much more input. Station MonitorShows GPS stations on a Google Map, displays last 10 minutes of data. Real Time RDAHMMDisplays RDAHMM results of last 10 minutes of GPS data in a Google map. Seismic Archive Query Portlet Google Map portlet that shows seismic events based on your query. * Fault Query PortletAllows you to query the QuakeTables fault data base for information on faults.* * Developed, needs to be converted into a portlet

RDAHMM Portlet: Main Navigation

RDAHMM Project Set Up

RDAHMM GRWS Query Interface

RDAHMM Results Page

Real Time RDAHMM Portlet

Station Monitor Portlet

ST_Filter Portlets

IU Web Service Work

Web Services Ant Execution ServiceGeneral purpose service for running applications. Wraps Apache Ant. RDAHMM ServiceBased on Ant service, but provides a RDAHMM- specific API. ST_Filter ServiceRuns ST_Filter, based on Ant service. Will have an ST_Filter-based WSDL. Web Feature ServiceGeneral purpose, OGC-compliant data service. Fault and seismic record data. Web Map ServiceMap-generating service. We have built both a OGC WMS and a Google tiling-style map server (currently Indiana specific but can be generalized). Context ServiceThis is a lightweight metadata management service that can be used to store user session data. Project provenance: how did I make this output file?

Sensor Grid Overview Publish/subscribe infrastructure for handling real time data.

Real-Time Services for GPS Observations Real-time data processing is supported by employing filters around publish/subscribe messaging system. The filters are small applications extended from a generic Filter class to inherit publish and subscribe capabilities. Input SignalOutput Signal Filter

Filter Chains

NaradaBrokering Topics

Real-Time positions on Google maps

Real-Time Station Position Changes

RDAHMM + Real-Time GPS Integration

SensorGrid Tests Galip Aydin – Zhigang Qi 11/16/2006

SensorGrid Tests Two Major Goals: System Stability and Scalability –Ensuring stability of the Filter Services for continuous operation. –Finding the maximum number of publishers (sensors) and clients that can be supported with a single broker. –Investigate if system scales for large number of sensors and clients.

Test Methodology The test system consists of a NaradaBrokering server and a three-filter chain for publishing, converting and receiving RYO messages. We take 4 timings for determining mean end-to-end delivery times of GPS measurements. The tests were run at least for 24 hours. Ttransfer = (T2 – T1) + (T4 – T3)

1- System Stability Test The basic system with three filters and one broker. The average transfer time shows the continuous operation does not degrade the system performance.

2 – Multiple Publishers Test We add more GPS networks by running more publishers. The results show that 1000 publishers can be supported with no performance loss. This is an operating system limit.

3 – Multiple Clients Test We add more clients by running multiple Simple Filters. The system can support as many as 1000 clients with very low performance decrease. Adding clients 1000 Clients

Extending Scalability The limit of the basic system appears to be 1000 clients or publishers. This is due to an Operating System restriction of open file descriptors. To overcome this limit we create NaradaBrokering networks with linking multiple brokers. We run 2 brokers to support 1500 clients. –Number of brokers can be increased indefinitely, so we can potentially support any number of publishers and subscribers. –Still have to test, of course.

4 – Multiple Brokers Test Messages published to first broker can be received from the second broker. We take timings on each broker. The results show that the performance is very good and similar to single broker test.

4 – Multiple Brokers Test 750 Clients

Test Results The RYO Publisher filter publishes 24-hour archive of the CRTN_01 GPS network which contains 9 GPS stations. The single broker configuration can support 1000 clients or networks (9000 stations) The system can be scaled up by creating NaradaBrokering broker networks. Message order was preserved in all tests.

Federating Map Servers Zao Liu, Marlon Pierce, Geoffrey Fox Community Grids Laboratory Indiana University

Integrating Map Servers Geographical Information Systems combine online dynamic maps and databases. Many GIS software packages exist GIS servers around state of Indiana –ESRI ArcIMS and ArcMap Server (Marion, Vanderburgh, Hancock, Kosciusco, Huntington, Tippecanoe) –Autodesk MapGuide (Hamilton, Hendricks, Monroe, Wayne) –WTH Mapserver™ Web Mapping Application (Fulton, Cass, Daviess, City of Huntingburg) based on several Open Source projects (Minnesota Map Server) Challenge: make 17 different county map servers from different companies work together. –92 counties in Indiana, so potentially 92 different map servers.

Considerations We assume heterogeneity in GIS map and feature servers. –GIS services are organized bottom-up rather than top-down. –Local city governments, 92 different county governments, multiple Indiana state agencies, inter-state (Ohio, Kentucky) consideration, federal government data providers (Hazus). –Must find a way to federate existing services. We must reconcile ESRI, Autodesk, OGC, Google Map, and other technical approaches. –Must try to take advantage of Google, ESRI, etc rather than compete. We must have good performance and interactivity. –Servers must respond quickly--launching queries to 20 different map servers is very inefficient. –Clients should have simplicity and interactivity of Google Maps and similar AJAX style applications.

Caching and Tiling Maps Federation through caching: –WMS and WFS resources are queried and results are stored on the cache servers. –WMS images are stored as tiles. These can be assembled into new images on demand (c. f. Google Maps). Projections and styling can be reconciled. We can store multiple layers this way. –We build adapters that can work with ESRI and OGC products; tailor to specific counties. Serving images as tiles –Client programs obtain images directly from our tile server. That is, don’t go back to the original WMS for every request. –Similar approaches can be used to mediate WFS requests. –This works with Google Map-based clients. –The tile server can re-cache and tile on demand if tile sections are missing.

Map Server Example Marion and Hancock county parcel plots and IDs are overlaid on IU aerial photographic images that are accessed by this mashup using Google Map APIs. We cache and tile all the images from several different map servers. (Marion and Hancock actually use different commercial software.)