1. A Partnership for the Next Generation of ESE Web Services Development Best Practices March 16, 2010 Presenters: Min-Gu Lee, Scott Campbell National Computer Center Application Hosting Web Conferences

2. slide 2 Copyright Lockheed Martin 5/17/2015 Agenda  Introduction  Web Services Architecture  An Inventory of EPA Web Services  Web Services Development  Web Services Deployment  Example of Web Services in an EPA Application  Questions

3. Introduction: Web Services Definitions slide 3 A software system designed to support interoperable machine-to-machine interaction over a network. - W3C Web Services provide an interface for a Service-Oriented Architecture (SOA), in which Web-based applications dynamically interact with other Web applications using open standards that include Extensible Markup Language (XML) running over Hypertext Transfer Protocol (HTTP), Universal Description, Discovery, and Integration (UDDI), and Apache Simple Object Access Protocol (SOAP). - State of Utah, Dept. of Tech. Services Web services are automated information services that are conducted over the Internet, using standardized technologies and formats/protocols that simplify the exchange and integration of large amounts of data over the Internet. - EPA, Environmental Information Exchange Network 5/17/2015

4. Introduction: Benefits slide 4  Application and data integration (interoperability)  Code Reusability  Cost Savings 5/17/2015

5. General Web Service Architecture slide 5 5/17/2015  Web Services in Terms of Roles Service Provider Service Requestor  Types of Web Services SOAP-based REpresentational State Transfer (RESTful): using HTTP methods (PUT, GET, DELETE, POST)  Web services in Terms of Protocol Stack Service Transport (including HTTP, Simple Mail Transfer Protocol (SMTP), File Transfer Protocol (FTP) XML Messaging (XML-RPC, SOAP) Service Description (Web Services Description Language (WSDL)) Service Discovery (UDDI)

6. Conceptual Web Service / SOA Architecture slide 6 5/17/2015

7. Conceptual Web Service / SOA Architecture slide 7 5/17/2015  Web Services Business Web Services Business functions that are exposed as Web services Utility Web Services Web services that provide general helper utilities for applications  Registry and Repository Registry Supports the publication, classification, discovery, and testing of enterprise Web Services Repository Stores the definitions and metadata of enterprise services and provides an integrated modeling environment for defining enterprise services

8. Conceptual Web Service / SOA Architecture slide 8 5/17/2015  Enterprise Service Bus (ESB) An architectural pattern for implementing a standard based functionality for transporting, routing, mediating, and orchestrating Extensible Markup Language (XML) messages in a secure manner  Federated Enterprise Service Bus Multiple ESB domains working together to form a single logical ESB infrastructure Addresses service governance, security and management concerns

9. Conceptual Web Service / SOA Architecture slide 9 5/17/2015  Business Process Modeling (BPM) BPEL (Business Process Execution Language) – A standard executable language for specifying interactions with Web Services BPMN (Business Process Modeling Notation) – A notation that specifies business process in a workflow and is understandable by all users.  Monitoring BAM (Business Activity Monitoring) – An enterprise solution offering real-time summary of business activity to managers and partners EM (Enterprise Monitoring) – Enterprise-wide monitoring of real-time operations and activities

10. An Inventory of EPA Web Services …. slide 10 5/17/2015

11. slide 11 Examples of EPA Web Services by Access Level EPA Extranet Services (through CDX) EPA Internal Services EPA Public Services Open Services by Vendors  44 data flows currently receive data from EPA, States, Tribes, and Industry.  Refer to http://epa.gov/cd x/projects/produc tion.htm for a complete list  Enterprise Content Management System (ECMS) Web services  Web Access Management (WAM) Web services  Oracle Collaboration (OCS) Web Services  Taxonomy Services  UVIndex  STOrage and RETrieval (STORET) Services  Microsoft Bing Map Web Services  Google Earth Service 5/17/2015

12. EPA Extranet Services via CDX …. slide 12 5/17/2015

13. Central Data Exchange (CDX) slide 13  Point of entry on the Environmental Information Exchange Network (Exchange Network) for environmental data submissions to the Agency  Partner organizations include: EPA Program Offices State and Local governments Industry Tribes  Cross-Media Electronic Reporting Regulation (CROMERR) Provides legal framework for electronic reporting to the EPA Reports submitted electronically to EPA must be submitted to CDX  CDX offers a full set of services including: Authentication and authorization Submission Validation Data archiving User support 5/17/2015

14. Examples of EPA Systems Using CDX Web Services slide 14  Air Quality System (AQS)  eBeaches  Electronic Inventory Update Rule (eIUR)  Facility Registry System  Grants.gov / Integrated Grants Management System (IGMS)  National Emissions Inventory (NEI)  Safe Drinking Water Information System (SDWIS)  Substance Registry System  Toxics Release Inventory (TRI) State Data Exchange  Water Quality Exchange (WQX) 5/17/2015 http://www.epa.gov/cdx/projects/production.htm The complete list of the CDX Production Data Flows can be found at:

15. Web Services Inventory CDX (Extranet) slide 15 Web Service Group MethodsFunction Network Security Services AuthenticateProvides the capability for the node or user to authenticate with the Network Authentication and Authorization Server (NAAS) and use the obtained Security Token to access services on the Network including data exchanges. AuthorizationDetermines if the service requestor is entitled to perform the operation, which can range from invoking the Web service to executing a certain part of its functionality (executing a stored procedure, for instance, using the Query method). Source: CDX Web Services Fact Sheet, http://www.epa.gov/cdx/about/webservicesfactsheet.pdf 5/17/2015

16. Web Services Inventory CDX (Extranet) slide 16 Web Service Group MethodsFunction Database Services QueryProvides the capability to perform a database query and return the result in XML format to the requestor. Queries are associated with specific schema/data flows supported by the Network. SolicitProvides the capability to perform an asynchronous database query and return the result in XML format to the requestor. Document Exchange Services SubmitProvides the capability to send one or multiple payloads (Documents) to the service provider. DownloadProvides the capability to retrieve one or multiple payloads from the service provider. Source: CDX Web Services Fact Sheet, http://www.epa.gov/cdx/about/webservicesfactsheet.pdf 5/17/2015

17. Web Services Inventory CDX (Extranet) slide 17 Web Service Group MethodsFunction Transaction Status Services Get StatusEach data exchange service is associated with a unique transaction. This Web Service provides the capability of querying the current state of the transaction. Notification Services NotifyEnables the requestor to notify a node about: Document availability Status, such as current status of a submission or service request Events such as security alerts, shutdown notices, and other network management notes. Administration Services Node PingProvides information about the accessibility of the node. Get ServicesAllows requesters to query services provided by a Network node. 5/17/2015 Source: CDX Web Services Fact Sheet, http://www.epa.gov/cdx/about/webservicesfactsheet.pdf

18. Web Services Inventory WQX via CDX (Extranet) slide 18 Web Service Group Web ServicesFunction WQX Data Services (Query/Solicit) WQX.GetActivityByParameters_v2.0Returns a collection of Activities and a count of the number of Results on each Activity. WQX.GetActivityGroupByParameters_v2.0Returns a collection of Activity Groups. WQX.GetBiologicalHabitatIndexByParameter s_v2.0 Returns a collection of Biological or Habitat Indices. WQX.GetDomainValueByElementName_v2.0Returns a list of domain values for the elementName passed in. 5/17/2015 Source: Water Quality Exchange, Flow Configuration Document, http://www.exchangenetwork.net/schema/WQX/2/WQX_FCD_v2.0.pdf

19. Web Services Inventory WQX via CDX (Extranet) slide 19 Web Service Group Web ServicesFunction WQX Data Services (Query/Solicit) WQX.GetMonitoringLocationByParameters_v 2.0 Returns a collection of Monitoring Locations. WQX.GetProjectByParameters_v2.0Returns a collection of Projects for the organization passed in. WQX.GetResultByParameters_v2.0Returns a collection of Activities and Results. WQX.GetTransactionHistoryByParameters_v 2.0 Returns a summary of the transactions processed by the WQX System. 5/17/2015 Source: Water Quality Exchange, Flow Configuration Document, http://www.exchangenetwork.net/schema/WQX/2/WQX_FCD_v2.0.pdf

20. Web Services Inventory Substance Registry System via CDX (Extranet) slide 20 Web Service Group MethodsFunction Data Processing Services (Submit) ListSubstanceAdditionAllows an authorized partner to add a substance that is already defined in SRS to a substance list that is also already defined in SRS. Substance Retrieval Data Publishing (Query) GetSubstanceByALTIDList Acronym Allows partners to select substances based upon an alternate identifier for the substance and whether the substance is a member of a specified list within SRS. GetSubstanceByCASAllows partners to select substances based upon the Chemical Abstracts Service (CAS) registry number associated with the substance in SRS. GetSubstanceByEPAIDAllows partners to select substances based upon the EPA identifier associated with the substance in SRS. GetSubstanceByInternalTr ackingNumber Allows partners to select substances based upon the SRS internal tracking number associated with the substance in SRS. GetSubstanceByNameAllows partners to select substances based upon a substance name for the substance. The substance name will match based upon a substance’s name, systematic name, or one of its synonym names. 5/17/2015 Source: Substance Registry System, Flow Configuration Document, http://www.exchangenetwork.net/schema/SRS/3/SRS_FCD_v3%200.doc

21. Web Services Inventory Substance Registry System via CDX (Extranet) slide 21 Web Service Group MethodsFunction Substance Retrieval Data Publishing (Query) GetSubstanceByTSNAllows partners to select substances based upon the taxonomic serial number associated with the substance in SRS. GetSubstanceMatchByNameAllows partners to determine whether a substance exists within SRS based upon a substance name for the substance. Substance Retrieval Data Publishing (Solicit) GetSubstancesByCASAllows partners to select substances based upon the CAS registry number associated with the substance in SRS. GetSubstancesByCategoryClassificationsAllows partners to select substances based upon the classifications associated with the substance in SRS. GetSubstancesByCharacteristicsAllows partners to select substances based upon the characteristics associated with the substance in SRS. 5/17/2015 Source: Substance Registry System, Flow Configuration Document, http://www.exchangenetwork.net/schema/SRS/3/SRS_FCD_v3%200.doc

22. Web Services Inventory Substance Registry System via CDX (Extranet) slide 22 Web Service Group MethodsFunction Substance Retrieval Data Publishing (Solicit) GetSubstancesByEPAIDsAllows partners to select substances based upon the EPA identifier associated with the substance in SRS. GetSubstancesByInternalTrackingNumberAllows partners to select substances based upon the SRS internal tracking number associated with the substance in SRS. GetSubstancesByListAcronymAllows partners to select all substances that are a member of the specified list in SRS. GetSubstancesByNamesExactAllows partners to select substances based upon a substance name for the substance. GetSubstanceCASByListAcronymAllows partners to select only the CAS registry numbers of all substances that are a member of the specified list in SRS. GetSubstanceListAcronymsByAcronymallows partners to determine substance list acronyms that exist in SRS. 5/17/2015 Source: Substance Registry System, Flow Configuration Document, http://www.exchangenetwork.net/schema/SRS/3/SRS_FCD_v3%200.doc

23. Web Services Inventory Facility Registry System via CDX (Extranet) slide 23 Web Service Group MethodsFunction FRS Data Services GetFacilityCount_v3.0 (Query) Partners may choose to implement this internally for costing purposes, so that over-large queries can be intercepted before they are run. GetFacilityList_v3.0 (Query or Solicit) Full set of parameters but returning on the lightest payload (FacilityIndex) to aid in performance. GetFacility_v3.0 (Solicit)Full set of parameters, also including Change Date, and returning a payload based on the full schema. GetFacilityInterest_v3.0 (Query) Full set of parameters but returning a payload of summary facility and environmental interest data (FacilityInterest) rather than the full schema, to aid in performance. 5/17/2015 Source: Facility Identification Flow Configuration Document, http://www.exchangenetwork.net/schema/FACID/3/FACID_FCD_v3.0_DRAFT.doc

24. Web Services Inventory Facility Registry System via CDX (Extranet) slide 24 Web Service Group Web ServiceFunction FRS Data Services GetFacilityByID_v3.0 (Query) Get detailed facility data for one facility. GetFacilityByChangeDat e_v3.0 (Solicit) Used to support the creation and maintenance of a replica set of facility data across Partners (i.e., data synchronization). GetDeletedFacilityByCh angeDate_v3.0 (Solicit) This uses the FacilityIndex schema to return of basic identification data about each facility that has been deleted. This would only be used if a Partner is maintaining a replica set of facility data. 5/17/2015 Source: Facility Identification Flow Configuration Document, http://www.exchangenetwork.net/schema/FACID/3/FACID_FCD_v3.0_DRAFT.doc

25. EPA Internal Services …. slide 25 5/17/2015

26. Web Services Inventory Web Access Management (Internal) slide 26 Web Service GroupMethodsFunction Authentication Manager Services Authenticate, validate, getUserProvides user registration and resource access requests. Self-Service FunctionschangePassword, forgotUserAllows users to manage account attributes (e.g., their own passwords) across managed resources. Provisioning Service Functions subscribeToGroup, unsubscribe from Group Process of making applications ready for users to use once self-registration info has been entered. Delegated Administration Service Functions getSelfRegistrationPendingReq uests, getSubscribeToGroupPendingR equests, approveSelfRegistrationPendin gRequest, addGroupMember, removeGroupMember Supports delegation of most administrative functions to any group and/or user. 5/17/2015 Source: Web Access Management Framework Integration Guide, Version 1.2, https://badger.epa.gov/iamfw/

27. Web Services Inventory Taxonomy Services (Internal) slide 27 Web Service Group MethodsFunction Synaptica Automated Services LoginUsers may log in to obtain a session key. Valid session keys can then be passed to all Synaptica web methods as authentication with the exception of Login. LogoutThis method expires a session key making it invalid. As a security measure, Logout should be called to release any session keys obtained by a user that will no longer be needed. ListTaskViews70This method returns a list of the taskview profiles a user has permission to. It has the exact same functionality as ListTaskviews, except for its input. ListVocabularies70This method returns a list of the vocabularies a user has permission to within a Taskview. ImportTerms70This method imports terms and relationships provided in valid Resource Description Framework / Simple Knowledge Organization System (RDF/SKOS) format as defined in "Synaptica_RDF_SKOS_specification_20061129.doc" 5/17/2015 Source: Synaptica Knowledge Management System, Synaptica Web Services API. 02/04/2008

28. Web Services Inventory Taxonomy Services (Internal) slide 28 Web Service Group MethodsFunction Synaptica Automated Services ExportVocabulary70Export an RDF/SKOS representation of one or more vocabularies from Synaptica. This method can be used for all general extracts from Synaptica. ValidateTerms70The method receives a list of terms to be validated within Synaptica. For each term, a descriptor and vocabulary name is supplied. ListRelatedTerms70This method returns relationships for a given term. The method request can specify a single relationship type or all relationships. GetSavedReport70This method returns the contents of files saved using the scheduled report functionality of the Synaptica Web Interface. SearchTerms70This method is used to search object classes for specific terms by various criteria exposed by the Alphabetical view report. This may be used to return entire object classes, or narrow results depending on the input parameters. 5/17/2015 Source: Synaptica Knowledge Management System, Synaptica Web Services API. 02/04/2008

29. Web Services Inventory Enterprise Content Management Services (Internal) slide 29 Web Service GroupMethodsFunction ERMAWebServiceGETSUMMARYDXLRequestConstructs summary XML header information and ECMS attributes. GETDXLRequestConstructs IBM Domino Extended Markup Language (DXL). GETMANUALDXLRequestConstructs Manual DXL. GETDXLBASE64RequestProcesses e-mail attachments. CONFIRMARCHIVEDRequestCreates lock icon after an e- mail is saved to E- mail Records Management Application (ERMA) repository. Documentum 6.5 Foundation Web Services Link to archive that contains EMC Documentum Foundation Services 6.5 SP1 Technical Publications: https://community.emc.com/servlet/JiveServlet/downloadBody/2820- 102-5- 10615/dfs_65sp1_pubs.zip;jsessionid=1FFB8433464B4BCAD6014CF 374E80BD7.node0 5/17/2015

30. EPA Public Services …. slide 30 5/17/2015

31. Web Services Inventory STORET (Public) slide 31 Web ServiceMethodsFunction Watershed Summary Web Service getOrganizationSummaryProvides organization and characteristic summary information aggregated at the watershed level or aggregated for a given organization within the watershed. getCharTypeCharacteristicSummary getCharacteristicSummary getOrganizationCharTypeSummary getOrganizationCharTypeCharacterisitic Summary getCharTypeSummary getWebServiceCatalog 5/17/2015 Source: STORET Web Services Consumption User Guide, http://www.epa.gov/storpubl/STORETWebServicesUserGuide2008_0205.pdf

32. Web Services Inventory STORET (Public) slide 32 Web ServiceMethodsFunction Station Catalog Web Service getWatershedOrgStationCharTypeSummar y Provides station summary information such as station name, period of record, and result counts by characteristic type. getStationCharTypeSummaryUsingXMLIn put getStationCharacteristicSummary getStationCharTypeSummaryUsingStringI nput getWebServiceCatalog getWatershedStationCharTypeSummary 5/17/2015 Source: STORET Web Services Consumption User Guide, http://www.epa.gov/storpubl/STORETWebServicesUserGuide2008_0205.pdf

33. Web Services Inventory STORET (Public) slide 33 Web ServiceMethodsFunction Project Catalog Web Service getProjectCountByLatLongprovides project summary information such as the period of record, characteristics monitored, and station names getProjectInfoByParameters getProjectCountByParameters getProjectInfoByLatLong 5/17/2015 Source: STORET Web Services Consumption User Guide, http://www.epa.gov/storpubl/STORETWebServicesUserGuide2008_0205.pdf

34. Web Services Inventory STORET (Public) slide 34 Web ServiceMethodsFunction Station Web Service getStationInfoProvides information for all monitoring locations within a given geographic area (geographic bounding box). (Note: The Station Web Service has been revised (as of February 8, 2008) into multiple web methods that provide enhanced data return.) getStationCharacteristicsForMap getWebServiceCatalog getStationCount getStationsForMap 5/17/2015 Source: STORET Web Services Consumption User Guide, http://www.epa.gov/storpubl/STORETWebServicesUserGuide2008_0205.pdf

35. Web Services Inventory STORET (Public) slide 35 Web ServiceMethodsFunction STORET Result Web Service getResultCountProvides direct access to STORET results. (Note: The Result Web Service has been revised (as of February 8, 2008) to provide for faster data retrieval. ) getActivityCount getResults 5/17/2015 Source: STORET Web Services Consumption User Guide, http://www.epa.gov/storpubl/STORETWebServicesUserGuide2008_0205.pdf

36. Web Services Inventory UV Index (Public) slide 36 Web ServiceMethodsFunction UV Index Web Service getUVIndexAlertByCityState Obtains the specified UV Index for the specified city and state. getUVIndexAlertByZipCodeObtains the specified UV Index for the specified zip code. 5/17/2015 Source: http://www.epa.gov/sunwise/uviresources.html#webservice

37. Web Services Inventory Open Services from Vendors slide 37  Microsoft Bing Map Services  Google Earth Services 5/17/2015

38. Web Services Development Process slide 38  Define system requirements  Search for existing services in EPA UDDI Registry and EPA Reusable Component Services (RCS)  EPA Pre-Production UDDI Registry (https://uddi.epacdxnode.net)https://uddi.epacdxnode.net  EPA Production UDDI Registry (https://uddi.epa.gov/)https://uddi.epa.gov/  Create an Interface Control Document (ICD) and WSDL 5/17/2015

39. Web Services Deployment Process slide 39  Design / develop application using WSDL  Test application, integrating with web services  Deploy at NCC  Conduct EPA Application Deployment Checklist (ADC)  Deploy to NCC Staging environment  Publish in EPA UDDI and update SoR Reusable Component Services (RCS)  Deploy application and web services to NCC Production environment 5/17/2015

40. Example: Use of WAM Web Services in an EPA Application slide 40  Application: Emergency Management Portal (EMP)  User Community: Office of Emergency Management (OEM) and other authorized personnel  Application Profile: The EMP represents a single point of entry to site-specific information, asset management tools, and emergency management information  Problem: Upon authentication to EMP, determine the user’s access privileges to EMP functionality  Solution: WAMFW Web Service call to determine the group membership for a user and map the group to an internal EMP role 5/17/2015

41. Example: Use of WAM Web Services in an EPA Application slide 41 5/17/2015 Source: Emergency Management Portal Security Document – Role and Group Management, Version 1.2, 12/09/2009 Determination of EMP Equipment Module Groups and Roles

42. slide 42 5/17/2015 QUESTIONS ……