1. 19 September 2006 NATO UNCLASSIFIED Deploying Web Services: Bringing NNEC To The Edge Presentation to MCC 2006 James Busch Principal Scientist NC3A The Hague

2. NATO UNCLASSIFIED2 Deploying Web Services: Bringing NNEC to the Edge Brief Intro to NNEC Brief Intro to SOA and Web Services Deployed Web Services Experiment Conclusions

3. NATO UNCLASSIFIED3 NNEC?  “It is a global, Web-enabled platform for multiple forms of collaboration. This platform enables individuals, groups, companies and universities anywhere in the world to collaborate - for the purposes of innovation, production, education, research, entertainment [and] war-making - like no creative platform before.” New York Times columnist Thomas L. Friedman on IT’s Effect on Globalization, 2006

4. NATO UNCLASSIFIED4 NNEC!  “Future operations will be more complex and multidimensional. The planning and execution of operations will... [require] truly interoperable forces. To support this end, the use of Alliance forces must change from a pattern of de-confliction to one of integration where emerging technologies and concepts, like network-enabled capability, are increasingly used.” NATO’s Strategic Commanders, 2004

5. NATO UNCLASSIFIED5 NNEC Challenge  Information comes from many sources and is needed by many different users. It must be:  Securely Shared  Commonly Understood  Integrated and Analyzed …but how? ?

6. NATO UNCLASSIFIED6 NNEC Feasibility Study Ambitions  Identify types of C2ISR capabilities required to ‘enable’ NATO Network Centric Operations  Based on agreed Missions, Concepts of Operations, involving use of Modular Force Structures such as the NRF (NATO Response Force)  Utilization of Capability Based Planning approach to identify ‘mix’ of National and NATO capabilities  Develop a Strategy and Roadmap for realization of a Networking and Information Infrastructure (NII)  To ‘enable’ required C2ISR capabilities and to support the broader needs of the Alliance.  Involves networking of national funded, NATO common funded and multi-national funded capabilities

7. NATO UNCLASSIFIED7 Main Technical Components of NII  Communications  Information and Integration Services  Information Assurance  Service Management

8. NATO UNCLASSIFIED8 Information Systems Recommendations for the NII  Service Oriented Architecture  Loose coupling, dynamic discovery, service oriented  Open and accepted standards  XML, SOAP, UDDI, WS-Security, etc.  “Federation of Systems” vice “System of Systems” Concept  Autonomous Systems  Federated Services  National Centric – no NATO “GIG”  Flexible level of national participation (CMM concept)  Incorporation of legacy systems  Interface to – as opposed to eliminate – existing capability  This includes new “Value Added” services such as REP  Metadata supported information sharing and exchange  XML as common language  Development of “ontologies” to support information sharing  Usage of NATO XML Registry

9. NATO UNCLASSIFIED9 Deploying Web Services: Bringing NNEC to the Edge Brief Intro to NNEC Brief Intro to SOA and Web Services Conclusions Deployed Web Services Experiment

10. NATO UNCLASSIFIED10 Service Oriented Architecture   Service Oriented Architecture is the future of system development: “…the single most important theme in modern application development is service oriented architecture…” – Gartner Group   Recommended in NNEC Feasibility Study!   Information is provided as a series of “Services” – which can be as granular as you like   A “Consumer” requests information from a “Producer” (Service) and receives a response with some kind of data   Benefits – –Rapidly integrate new and existing systems into the ‘sharing network’  maximize interoperability. – –Utilize ‘open source’ and COTS that support ‘open standards’  minimize cost and reduce/eliminate vendor dependencies. – –Securely share information across various security levels amongst various coalition partners. – –Reduced integration costs and time compared with current approaches and technologies.   Web Services – an instantiation of SOA – holds promise for solving traditional integration problems using commonly available technologies – XML, HTTP, TCP/IP, etc. – and enabling integration with minimal or no changes to underlying (new or existing) systems

11. NATO UNCLASSIFIED11 What is Service Oriented Architecture? Publish Data and applications available for use accessible via services. Metadata added to services based on producer’s format. Service Producer Describes content using metadata Posts metadata in catalogs for discovery Exposes data and applications as services Discover Request/Invoke Searches metadata catalogs to find data services Analyzes metadata search results found Pulls selected data based on metadata understanding Automated search of data services using metadata. Pulls data of interest. Based on producer registered format and definitions, translates into needed structure. Service Consumer Service Enabled Infrastructure Messaging Services Monitoring Services Registry Services Security Services Transformation Services Data Services

12. NATO UNCLASSIFIED12 SOA  Web Services  SOA is an enterprise architecture  Web Services is an implementation of that architecture…  …following a common suite of standards (W3C, OASIS, etc.)  XML for data  SOAP for messaging  HTTP over TCP for transport  UDDI for registering services  WSDL for describing services  Can be used to open a controlled “window” into an existing application (aka the Web Service interface)

13. NATO UNCLASSIFIED13 Web Services – How they work Registry (Discovery Service) Security Service Service (Information Provider) Network Services come “on line” (join the network) Web Services Interface Web Services Interface Web Services Interface

14. NATO UNCLASSIFIED14 Deploying Web Services: Bringing NNEC to the Edge Brief Intro to NNEC Brief Intro to SOA and Web Services Conclusions Deployed Web Services Experiment

15. NATO UNCLASSIFIED15 Web Services Pattern 1: Synchronous Request-Response  Consumer makes request from Service and waits for response  No state (persistence) – Service “forgets” about Consumer as soon as response is sent  Pluses  Simple, most basic model  Easy to implement (any web browser can be a client)  Loose coupling between Client and Service  Minuses  Client waits on Service  Data is only received when asked for (not good for time- dependent data)

16. NATO UNCLASSIFIED16 Web Services Pattern 1: Synchronous Request-Response Useful for: Services where the information is not time- dependent (e.g. document repositories) Situations where Clients only want data when it asks When simplicity is important Situations where Client and Service(s) have no knowledge of each other

17. NATO UNCLASSIFIED17 Web Services Pattern 2: Asynchronous “Push” (aka Publish-Subscribe)  Consumer asks Service to send data whenever it is available (“subscribes” to Service)  Consumer implements a “listener”; Service implements capability to initiate messages (not just respond)  Pluses  Data is “pushed” out as soon as it is available, and only when it changes  Client doesn’t have to wait on service  Less back-and-forth between client and server? (more on this later…)  Minuses  Both Consumer and Service must implement capability beyond the basic  Consumer and Service are forced to “know” about each other

18. NATO UNCLASSIFIED18 Web Services Pattern 2: Asynchronous “Push” Useful for: Service offering very time- sensitive information (e.g. (near)real-time tracks) Client who wants new information as soon as it is available without having to ask for it Reducing the number of queries between Client and Service?

19. NATO UNCLASSIFIED19 The Overhead of Web Services  “XML/SOAP Adds So Much Overhead!” Payload goes here… Payload goes here…  The Disadvantage Is:  Lots of overhead from all the tags, headers, white space  The Benefits Are:  Structured information via XML (vital for information sharing, interoperability)  SOAP Envelope used for WS-Security, WS-Federation, etc.

20. NATO UNCLASSIFIED20 The Overhead of Web Services (2)  XML “Efficiency”  Huge advances in last few years  W3C Efficient XML Interchange Working Group working on standard for efficient XML transmission  Reduce XML documents by factor of 100  Maintain all the benefits of XML, mostly lose the drawback  But…  Digital Signatures and Encryption remain problematic  Add more overhead for security  Don’t compress well  XML is still not appropriate for every situation  Very small messages  Streaming data

21. NATO UNCLASSIFIED21 The Network Effect of Web Services Two factors affect the performance of Web Services  Throughput: The number of requests handled in a given time period by the service (measured on server side)  Latency: The round-trip time between sending the request and receiving a response (throughput + network transit). Impacted by:  Amount of bandwidth available (real and effective)  Message size/composition  Additional overhead imposed by HTTP

22. NATO UNCLASSIFIED22 Web Services at the “Edge”?  Key component in transformed NATO is “lightweight” (tactical) client  Mobile user  Deployed CJTF  So what are effects on Web Service performance of  Reduced network bandwidth  Network congestion  Larger/smaller volumes of data

23. NATO UNCLASSIFIED23 EPW1038 at CWID 2006  Experiment designed to test variety of Web Service calls in a variety of network conditions  Query returning small amounts of data (small request, small response)  Query returning large amounts of data (small request, large response)  Publish large amounts of data (large request, small response)  Subscribe to “pushed” track and weather information (potentially very large amounts of data)  Related to RTO-061 experiment described yesterday  Where are inefficiencies and what (if anything) can be done?

24. NATO UNCLASSIFIED24 Network 1: Local LAN Clients and all Services local to CWID (Norway) Network capacity of 100 Mbps

25. NATO UNCLASSIFIED25 Network 2: Remote Services via CFBLNet Clients local to CWID (Norway) Services remote at NC3A (Netherlands) accessed via CFBLNet backbone Network capacity of +/- 2 Mbps

26. NATO UNCLASSIFIED26 Network 3: Remote Services via UMTS Clients local to CWID (Norway) Services remote at NC3A (Netherlands) accessed via UMTS (mobile phone network) Network capacity of 30 - 300 Kbps

27. NATO UNCLASSIFIED27 Network Instrument Observer

28. NATO UNCLASSIFIED28 Web Service Simple Query Client Service REQUEST RESPONSE

29. NATO UNCLASSIFIED29 Web Services Communications on a Simple Query Client Service Initiation Acknowledgement (1 st phase handshake) Acknowledgement (2 nd phase handshake) REQUEST Acknowledgment RESPONSE Acknowledgment Termination Termination Acknowledgement Fact: Network performance is strongly influenced by the number of packets! Conclusion: Web Services back-and-forth adds significant traffic in the form of many small packets. Reducing these small messages helps increase efficiency.

30. NATO UNCLASSIFIED30 Network Capacity Effects on Web Service performance Web Services worked well until the worst bandwidth/message size configuration. Performance degradation is not linear but logarithmic; trends for large and small messages are nearly identical. Conclusion: Until the “pipe” becomes very small (relative to message size) the size of the message has less impact than the number of packets.

31. NATO UNCLASSIFIED31 Overhead of HTTP/SOAP on a Simple Query Bytes of Server Response Total Bytes of “conversation” Percent of Bytes that are “overhead” Small Query 675192065% Large Query 4291553625% Web Services (with SOAP and HTTP headers, plus the various small messages in the conversation) can add significant overhead Conclusion: The impact is much less on larger messages than on small. Configure for larger packet sizes where possible.

32. NATO UNCLASSIFIED32 Web Service Publish-Subscribe Client Service Initiation Acknowledgement (1 st phase handshake) Acknowledgement (2 nd phase handshake) REQUEST (“Subscribe”) Acknowledgment RESPONSE Acknowledgment RESPONSE Acknowledgment RESPONSE Acknowledgment RESPONSE Acknowledgment RESPONSE Acknowledgment

33. NATO UNCLASSIFIED33 Overhead of Multicast (“push”) vs. Request-Response Sample Size (Bytes) of Server Response Packets in “conversation” Approx. total Bytes in “conversation” Percent of Bytes that are “overhead” Synchronous10009230056% Asynchronous1000 3 (after subscribe) 150033% When data is “push”ed asynchronously, there are less packets in the conversation – generally an initiation, the message itself, and an acknowledgment from the recipient (client). Conclusion: “Push” of data asynchronously reduces HTTP/SOAP overhead because there are less messages in the “conversation”.

34. NATO UNCLASSIFIED34 Deploying Web Services: Bringing NNEC to the Edge Brief Intro to NNEC Brief Intro to SOA and Web Services Conclusions Deployed Web Services Experiment

35. NATO UNCLASSIFIED35 Conclusions  NNEC FS has suggested an NII architecture featuring  Service Oriented Architecture (implemented as Web Services where feasible)  Loosely coupled, dynamically discoverable systems (services)  Extremely valuable interoperability tool in the “enterprise” environment (strategic, operational)  On-going work already moving in this direction (including operationally)  Tests showed Web Services remained functional at low network capabilities  Except once message volume far exceeded available bandwidth  Appropriate for really low bandwidth (highly tactical) settings? Still uncertain…  Number of packets on network had bigger impact than larger request or response messages  Minimize the HTTP “conversations” where possible  Tweak HTTP settings (e.g. MTU) to enable fewer, larger packets  Eliminate superfluous messages (e.g. DNS pings) from network  Multi-cast (i.e. “push”) Web Services have a network performance advantage over request-response  Not appropriate for all situations

36. NATO UNCLASSIFIED36 Questions ?

37. NATO UNCLASSIFIED37 Contacting NC3A NC3A Brussels Visiting address: Bâtiment Z Avenue du Bourget 140 B-1110 Brussels Telephone +32 (0)2 7074111 Fax +32 (0)2 7078770 Postal address: NATO C3 Agency Boulevard Leopold III B-1110 Brussels - Belgium NC3A The Hague Visiting address: Oude Waalsdorperweg 61 2597 AK The Hague Telephone +31 (0)70 3743000 Fax +31 (0)70 3743239 Postal address: NATO C3 Agency P.O. Box 174 2501 CD The Hague The Netherlands