1. More than you probably wanted to know about NWIS and NWISWeb U.S. Department of the Interior, U.S. Geological Survey Kenneth J. Lanfear USGS

2. n Transactional Data entry Quality assurance n Access by USGS only n 1980’s and updated n 48 installations at local offices n Database of record n Large subset of NWIS Updated only from NWIS n Public access n 1995 n Organized primarily around station NWISWebNWIS

3. What’s in NWIS but not NWISWeb? n Unit values (except current) 1 daily value = 96 unit values Working on providing this. Older archives are on paper tapes and probably never will become widely available! n Rating curves (measurements available) n Local offices can restrict some data Quality-assurance issues Cooperator restrictions

4. Components of NWISWeb n Real-time n Site Inventory n Surface Water Daily values Statistics Peak flows Field measurements n Ground Water Levels n Water Quality

5. Selection Choices within NWISWeb n Site location State, county, hydrologic unit, lat-lon box NOT: Map … coming soon! NOT: NHD-id (Reach number) n Site identifier Name, number, agency code, etc. n Site attribute Site type, drainage area, well depth, etc. n Data attribute Update time, period of record NOT: Data values!

6. Site Location: The dirty little secret about accuracy n Most sites were established before GPS and GIS. n Location refers to gage house n Reviewed 24,055 locations 73% automatically matched to NHD with no problem 17% changed gage-house loc. t Avg. change = 1,100 m Est. 1903 LOCATION.--Lat 43° 51'30", long 110° 35'09", in SW1/4 SE1/4 sec.18, T.45 N., R.114 W., Teton County, Grand Teton National Park, Hydrologic Unit 17040101, on left bank 1,000 ft downstream from Jackson Lake Dam, 4.1 mi west of Moran, and at mile 988.7.

7. Every NWISWeb retrieval is a “cgi GET” request. n The entire request is expressed within a URL. n Can bookmark. n You can deduce the parameters needed to make a different request. http://nwis.waterdata.usgs.gov/nwis/discharge?site_no=01613000

8. Current NWISWeb Data Mining User Workstation Or Computer NWISWeb Server 1. Request for files of data 2. File or files are returned, usually in RDB format 3. User written program pulls out the wanted data from NWISWeb file 4. Data are stored or merged with other data for a particular use Writing and maintaining a customized program is very time consuming to our users. In addition if NWISWeb changes the format of the data, user programs can break. Workstation Application

9. NWISWeb already has a web service … sort of! n Request stations in a lat/lon box: http://nwis.waterdata.usgs.gov/nwis/inventory? nw_longitude_va=1120100&nw_latitude_va=16 4500&se_longitude_va=1120000&se_latitude_v a=164000&coordinate_format=dms&format=sit efile_output&sitefile_output_format=xml&colu mn_name=agency_cd&column_name=site_no &column_name=dec_lat_va&column_name=de c_long_va&column_name=coord_datum_cd&c olumn_name=alt_va&column_name=alt_datum _cd&column_name=station_type_cd&column_ name=rt_bol&column_name=discharge_begin_ date&column_name=discharge_end_date&colu mn_name=discharge_count_nu&column_name =peak_begin_date&column_name=peak_end_d ate&column_name=peak_count_nu&column_n ame=qw_begin_date&column_name=qw_end_ date&column_name=qw_count_nu&column_na me=gw_begin_date&column_name=gw_end_d ate&column_name=gw_count_nu&list_of_sear ch_criteria=lat_long_bounding_box n NWISWeb returns file

10. n User has to know how to construct the URL, unique to NWISWeb. n No schema to explain what the elements mean or how they’re related! n Error handling is non-standard. n None of these problems are insurmountable to a determined user. They just add to the costs of doing business with NWISWeb. Not really a web service

11. Retrieving the Site File in 1-degree blocks > 20,000 stations in 1-degree block. Texas restricts many wells which need QA.

12. XML Schemas precisely define n What information is needed to make a request n What will be returned By a successful request By a failed request

13. USGS Water Schemas n Need to define “official” XML schemas for describing USGS hydrological data n Schemas need to be carefully thought out Once widely used they will be difficult to change n Ideally schemas should first be coordinated with other agencies and groups that publish hydrological information

14. Example Web Service NWISWeb Internet cuahsi.com Database cuahsi.com Web Server Users sees USGS data on cuahsi.com Web site Other cuahsi.com data sources How much of NWISWeb is stored by 3 rd party and how much retrieved as needed?

15. Business Value of Web Services n Expands the ways that USGS Water Data can be used by the public. n Makes integrating USGS Water Data into other products much simpler. n Creates new markets for USGS Water Data. n Provides a means for the evolutionary reengineering of NWISWeb. Bottom line: We see CUAHSI as a distribution channel, not a competitor!

16. Business Model for NWISWeb Data n USGS collects data to support its own mission. USGS scientists and 700 cooperators n NWISWeb distributes data to everybody. Marginal cost to users is near zero. n Public uses data and gains benefits. Quality of life, higher GNP n Elected officials support USGS budget. We have CUAHSI, so why do we need the USGS?

17. How to move to web services: Current USGS thinking – not a promise! n Part One (FY05): Create Station Web Service Prototype Completed February 2005 Need to give additional demonstrations to other interested parties n Part Two (Proposed FY06): Define XML schemas for WRD hydrological data Elicit and document requirements from internal, government and external users Refine “best fit” web services technologies for WRD through further investigation and prototyping Internally publish and use some test web services as a learning experience n Part Three (FY07 and beyond) Allocate funding Start a limited “beta” service Refine technologies and infrastructure for production use Deploy production web services

18. Summary n NWISWeb is the public outlet for NWIS. Accurate transformation of NWIS data n USGS business model supports third- party distribution of NWISWeb data. n USGS is actively studying web services. Need advice/coordination on schemas Implementation depends on funding