1. © 2016 A. Haeberlen, Z. Ives CIS 455/555: Internet and Web Systems 1 University of Pennsylvania Web services March 21, 2016

2. © 2016 A. Haeberlen, Z. Ives Announcements HW2MS2 is due today! HW3 and Final Project handouts will be available soon Once the final project handout is available, please start thinking about project teams! Reading: Tanenbaum Chapters 4.2 and 10.3 SOAP and WSDL tutorials (links on course web page) 2 University of Pennsylvania

3. © 2016 A. Haeberlen, Z. Ives What is a web service? Intuition: An application that is accessible to other applications over the web Examples: Google Search, Google Maps API, Facebook Graph API, eBay APIs, Amazon Web Services,... 3 University of Pennsylvania Alice Bob Charlie Map service (used by Alice) Web page combines data from different sources ('mashup')

4. © 2016 A. Haeberlen, Z. Ives A more detailed definition Key elements: Machine-to-machine interaction Interoperable (with other applications and services) Machine-processable format Key technologies: SOAP (and also REST, both of which we've already seen) WSDL (Web Services Description language; XML-based) 4 University of Pennsylvania "A Web service is a software system designed to support interoperable machine-to-machine interaction over a network. It has an interface described in a machine-processable format (specifically WSDL). Other systems interact with the Web service in a manner prescribed by its description using SOAP messages, typically conveyed using HTTP with an XML serialization in conjunction with other Web-related standards." http://www.w3.org/TR/ws-arch/

5. © 2016 A. Haeberlen, Z. Ives Plan for today Remote Procedure Calls Abstraction Mechanism Stub-code generation Web services REST SOAP WSDL JSON 5 University of Pennsylvania NEXT

6. © 2016 A. Haeberlen, Z. Ives Motivation for RPCs Coding your own messaging is hard Example: Look up a name on our directory server Assemble the message at the sender, parse at the receiver... Need to think carefully about message flow etc. Can we hide this in the programming language or middleware? Similar strategy works great for many other hard or cumbersome tasks, e.g., memory management Wouldn't it be nice if we could simply call a function lookup(name) in the client code, and it executes remotely on the name server? That is the abstraction provided by Remote Procedure Calls 6 University of Pennsylvania

7. © 2016 A. Haeberlen, Z. Ives The intuition behind RPCs 7 University of Pennsylvania void foo() { int x, y;... x = bar(45, &y, false);... } void bar(int a, int *b, bool c) {... if (!c) *b = a + 17;... } Stack 45 &y false retaddr y Machine A Machine B "Message"

8. © 2016 A. Haeberlen, Z. Ives 8 Remote Procedure Calls Remote procedure calls have been around forever Goes back to at least 1976; Birrell/Nelson paper is from 1984 Implementation examples: COM+, CORBA, DCE, Java RMI,... An RPC API defines a format for: Initiating a call on a given server, generally in a reliable way At-most-once, at-least-once, exactly-once semantics Sending parameters (marshalling) to the server Receiving a return value - may require marshalling as well Different language bindings may exist Java client can call C++ server, Fortran client can call Pascal server, … RPC calls typically are synchronous Caller blocks until response is received from callee Exception: One-way RPCs University of Pennsylvania

9. © 2016 A. Haeberlen, Z. Ives 9 RPC visualized working request function executing server waits for next request client blocked (waiting for response) RPC Server RPC Client University of Pennsylvania response client continues server is busy

10. © 2016 A. Haeberlen, Z. Ives 10 How RPC generally works You write an application with a series of functions Some of these functions will be distributed remotely You call a stub-code generator, which produces A client stub, which emulates each function F: Marshals all the parameters and produces a request message Opens a connection to the server and sends the request Receives response, unmarshals+returns F’s return values, status A server stub, which emulates the caller on the server side: Receives a request for F with parameters Unmarshals the parameters, invokes F Takes F’s return status (e.g., protection fault), return value, marshals it, produces a response, and sends it back to the client Waits for the next request (or returns to the server loop) University of Pennsylvania

11. © 2016 A. Haeberlen, Z. Ives 11 Passing value parameters Steps involved in doing remote computation through RPC 2-8 University of Pennsylvania

12. © 2016 A. Haeberlen, Z. Ives 12 RPC components Generally, you need to write: Your function, in a compatible language An interface definition, analogous to a C header file, so other people can program for F without having its source Includes annotations for marshalling, e.g., [in] and [out] Special interface definition languages (IDLs) exist for this Stub-code generator takes the interface definition and generate the appropriate stubs (In the case of Java, RMIC knows enough about Java to run directly on the source file) The server stubs will generally run in some type of daemon process on the server Each function will need a globally unique name or GUID University of Pennsylvania

13. © 2016 A. Haeberlen, Z. Ives An example 13 University of Pennsylvania module StockObjects { struct Quote { string symbol; long at_time; double price; long volume; }; exception Unknown{}; interface Stock { // Returns the current stock quote. Quote get_quote() raises(Unknown); // Sets the current stock quote. void set_quote(in Quote stock_quote); // Provides the stock description, e.g. company name. readonly attribute string description; }; interface StockFactory { Stock create_stock(in string symbol, in string description ); }; }; http://java.sun.com/developer/onlineTraining/corba/corba.html

14. © 2016 A. Haeberlen, Z. Ives 14 Pass-by-value vs. pass-by-reference R1 essentially represented as (server,object) Result: Can pass references as parameters to RPCs University of Pennsylvania

15. © 2016 A. Haeberlen, Z. Ives 15 What are the hard problems with RPC? Resolving different data formats between languages (e.g., Java vs. Fortran arrays) Reliability, security Finding remote procedures in the first place Efficiency Extensibility/maintainability (Some of these might look familiar from when we talked about data exchange!) University of Pennsylvania

16. © 2016 A. Haeberlen, Z. Ives Plan for today Remote Procedure Calls Abstraction Mechanism Stub-code generation Web services REST SOAP WSDL JSON 16 University of Pennsylvania NEXT

17. © 2016 A. Haeberlen, Z. Ives 17 Web services Goal: Provide an infrastructure for connecting components, building applications in a way similar to hyperlinks between data Concept of a mashup It’s another distributed computing platform for the Web Goal: Internet-scale, language-independent, upwards- compatible where possible This one is based on many familiar concepts Standard protocols: HTTP Standard marshalling formats: XML-based, XML Schemas All new data formats are XML-based University of Pennsylvania

18. © 2016 A. Haeberlen, Z. Ives 18 The “Standard” for Web Services Three parts: 1. “Wire” / messaging protocols Data encodings, RPC calls or document passing, etc. We will discuss: SOAP and REST 2. Describing what goes on the wire Schemas for the data We have already discussed: XML Schema 3. “Service discovery” Means of finding web services (Historical example: UDDI) University of Pennsylvania

19. © 2016 A. Haeberlen, Z. Ives REST and SOAP Example: Access AWS from your program Example: Launch an EC2 instance, store a value in S3,... Simple Object Access protocol (SOAP) Not as simple as the name suggests XML-based, extensible, general, standardized, but also somewhat heavyweight and verbose Representational State Transfer (REST) Much simpler to develop than SOAP Web-specific; lack of standards 19 University of Pennsylvania

20. © 2016 A. Haeberlen, Z. Ives Representational State Transfer (REST) One example of a messaging protocol Not really a standard – a style of development Data is represented in XML, e.g., with a schema Function call interface uses URIs Server is to be stateless And the HTTP request type specifies the operation e.g., GET http://my.com/rest/service1http://my.com/rest/service1 e.g., POST http://my.com/rest/service1 {body} adds the body to the servicehttp://my.com/rest/service1 20 University of Pennsylvania

21. © 2016 A. Haeberlen, Z. Ives Example: REST 21 University of Pennsylvania https://sdb.amazonaws.com/?Action=PutAttributes &DomainName=MyDomain &ItemName=Item123 &Attribute.1.Name=Color&Attribute.1.Value=Blue &Attribute.2.Name=Size&Attribute.2.Value=Med &Attribute.3.Name=Price&Attribute.3.Value=0014.99 &AWSAccessKeyId= &Version=2009-04-15 &Signature=[valid signature] &SignatureVersion=2 &SignatureMethod=HmacSHA256 &Timestamp=2014-01-25T15%3A01%3A28-07%3A00 Success f6820318-9658-4a9d-89f8- b067c90904fc 0.0000219907 Sample requestSample response Source: http://awsdocs.s3.amazonaws.com/SDB/latest/sdb-dg.pdf Invoked method Parameters Credentials Response elements

22. © 2016 A. Haeberlen, Z. Ives The 'protocol stacks' of web services Enhanced + expanded from a figure from IBM’s “Web Services Insider”, http://www.ibm.com/developerworks/webservices/library/ws-ref2/index.html Other extensions MTOM / SOAP Attachments WS-Security, SAML WS-AtomicTransaction, WS-Coordination SOAP, XML-RPC XML XML Schema Service Description (WSDL) Service Capabilities (WS-Capability) Message Sequencing Orchestration (WS-BPEL) Inspection Directory (UDDI) Wire Format Stack Discovery Stack Description Stack WS-Addressing High-level state transition + messaging diagrams between modules 22 University of Pennsylvania

23. © 2016 A. Haeberlen, Z. Ives 23 Simple Object Access Protocol (SOAP) Another example of a messaging protocol XML-based format for passing parameters Has a SOAP header and body inside an envelope Has a defined HTTP binding (POST with content-type of application/soap+xml) A companion SOAP Attachments encapsulates other (MIME) data The header defines information about processing: encoding, signatures, etc. It’s extensible, and there’s a special attribute called mustUnderstand that is attached to elements that must be supported by the callee The body defines the actual application-defined data University of Pennsylvania Envelope Header Body

24. © 2016 A. Haeberlen, Z. Ives 24 Making a SOAP Call To execute a call to service PlaceOrder: POST /PlaceOrder HTTP/1.1 Host: my.server.com Content-Type: application/soap+xml; charset=“utf-8” Content-Length: nnn … University of Pennsylvania

25. © 2016 A. Haeberlen, Z. Ives 25 SOAP Return Values If successful, the SOAP response will generally be another SOAP message with the return data values, much like the request If failure, the contents of the SOAP envelop will generally be a Fault message, along the lines of: SOAP-ENV:Client Could not parse message … University of Pennsylvania

26. © 2016 A. Haeberlen, Z. Ives Example: SOAP envelope 26 University of Pennsylvania <SOAP-ENV:Envelope xmlns:SOAP-ENV='http://schemas.xmlsoap.org/soap/envelope/' xmlns:SOAP-ENC='http://schemas.xmlsoap.org/soap/encoding/' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xmlns:xsd='http://www.w3.org/2001/XMLSchema'> a1 2 a2 4 domain1 eID001 2009-04-15 4c68e051-fe45-43b2-992a- a24017ffe7ab 0.0000219907 Sample request Sample response Source: http://awsdocs.s3.amazonaws.com/SDB/latest/sdb-dg.pdf (

27. © 2016 A. Haeberlen, Z. Ives Plan for today Remote Procedure Calls Abstraction Mechanism Stub-code generation Web services REST SOAP WSDL JSON 27 University of Pennsylvania NEXT

28. © 2016 A. Haeberlen, Z. Ives 28 How do we declare functions? Remember interface definition languages? CORBA IDL, DCE IDL,... In Java, the interface uses the same language as the Java code Web Services Description Language (WSDL) is the interface definition language for web services Defines notions of protocol bindings, ports, and services Generally describes data types using XML Schema University of Pennsylvania

29. © 2016 A. Haeberlen, Z. Ives A WSDL Service Service Port PortType Operation PortType Operation PortType Operation Binding 29 University of Pennsylvania

30. © 2016 A. Haeberlen, Z. Ives 30 Web service terminology Service: The entire Web Service Port Type: Abstract grouping of operations, i.e. a class Port: Maps a set of port types to a transport binding (a protocol, frequently SOAP, COM, CORBA, …) Operation: The type of operation – request/response, one-way Input message and output message; maybe also fault message Types: The XML Schema type definitions University of Pennsylvania

31. © 2016 A. Haeberlen, Z. Ives WSDL example 31 University of Pennsylvania definition of types...... definition of a message.. definition of a port..... definition of a binding.. Uses XML Schema http://www.w3schools.com/wsdl/

32. © 2016 A. Haeberlen, Z. Ives 32 JAX-RPC: Java and web services Java API for XML-based RPCs To write JAX-RPC web service “endpoint”, you need two parts: An endpoint interface – this is basically like the IDL statement An implementation class – your actual code http://www.ibm.com/developerworks/xml/library/x-tipjaxrpc/ University of Pennsylvania public interface BookQuote extends java.rmi.Remote { public float getBookPrice(String isbn) throws java.rmi.RemoteException; } public class BookQuote_Impl_1 implements BookQuote { public float getBookPrice(String isbn) { return 3.22; }

33. © 2016 A. Haeberlen, Z. Ives Different options for calling The conventional approach is to generate a stub, as in the RPC model described earlier WSDL-to-Java code generator produces, among other things, a Service Endpoint Interface (SEI) - the 'client stub' You can also dynamically generate the call to the remote interface, e.g., by looking up an interesting function to call The “DII” (Dynamic Invocation Interface) method allows you to assemble the SOAP call on your own Create a Call object, configure it with parameters, return type, binding information, endpoint,..., then invoke the call 33 University of Pennsylvania

34. © 2016 A. Haeberlen, Z. Ives Creating a Java web service A compiler called wscompile is used to generate your WSDL file and stubs You need to start with a configuration file that says something about the service you’re building and the interfaces that you’re converting into Web Services <service name="StockQuote" targetNamespace="http://example.com/stockquote.wsdl" typeNamespace="http://example.com/stockquote/types" packageName="edu.upenn.cis455"> <interface name="edu.upenn.cis455.StockQuoteProvider" servantName="edu.upenn.cis455.StockQuoteServiceImpl"/> 34 University of Pennsylvania

35. © 2016 A. Haeberlen, Z. Ives 35 Starting a WAR The Web Service version of a Java JAR file is a Web Archive, WAR There’s a tool called wsdeploy that generates deployable WAR files from 'raw' WAR files Input WAR contains endpoint interface, implementation class, web.xml and a descriptor called jaxrpc-ri.xml wsdeploy edits web.xml & calls wscompile to generate WSDL + classes Generally this will automatically be called from a build tool such as Ant Finally, you may need to add the WAR file to the appropriate location in Apache Tomcat (or WebSphere, etc.) and enable it For a detailed example, see http://www.oracle.com/technetwork/java/webservicespack-jsp-140788.html University of Pennsylvania

36. © 2016 A. Haeberlen, Z. Ives Plan for today Remote Procedure Calls Abstraction Mechanism Stub-code generation Web services REST SOAP WSDL JSON 36 University of Pennsylvania NEXT

37. © 2016 A. Haeberlen, Z. Ives The Facebook API What you can do: Read data from profiles and pages Navigate the graph (e.g., via friends lists) Issue queries (for posts, people, pages,...) Add or modify data (e.g., create new posts) Get real-time updates, issue batch requests,... How you can access it: Graph API FQL (until v2.0) Legacy REST API 37 University of Pennsylvania

38. © 2016 A. Haeberlen, Z. Ives JSON Another standard for data interchange "JavaScript Object Notation"; MIME type application/json Basically legal JavaScript code; can be parsed with eval() Caution: Security! Often used in AJAX-style applications Data types: Numbers, strings, booleans, arrays, "objects" 38 University of Pennsylvania { "firstName": "John", "lastName": "Smith", "age": 25, "address": { "streetAddress": "21 2nd Street", "city": "New York", "state": "NY", "postalCode": 10021 }, "phoneNumber": [ { "type": "home", "number": "212 555-1234" }, { "type": "fax", "number": "646 555-4567" } ] } Array (ordered sequence of values; can be different types) "Object": Unordered collection of key-value pairs

39. © 2016 A. Haeberlen, Z. Ives The Graph API Requests are mapped directly to HTTP: https://graph.facebook.com/(identifier)?fields=(fieldList) Response is in JSON 39 University of Pennsylvania { "id": "1074724712", "age_range": { "min": 21 }, "locale": "en_US", "location": { "id": "101881036520836", "name": "Philadelphia, Pennsylvania" } }

40. © 2016 A. Haeberlen, Z. Ives The Graph API Uses several HTTP methods: GET for reading POST for adding or modifying DELETE for removing IDs can be numeric or names /1074724712 or /andreas.haeberlen Pages also have IDs Authorization is via 'access tokens' Opaque string; encodes specific permissions (access user location, but not interests, etc.) Has an expiration date, so may need to be refreshed 40 University of Pennsylvania

41. © 2016 A. Haeberlen, Z. Ives Older API options Facebook Query Language (FQL) SQL-style queries over the data provided via the Graph API Example: SELECT name FROM user WHERE uid=me() Supports 'multi-queries' (JSON-encoded dictionary of queries) 72 different tables are available for querying Supported until v2.0 of the API (current is v2.5) Legacy REST API See description of REST in the previous lecture Now deprecated Large number of methods available Examples: friends.get, comments.add, status.set, video.upload,... 41 University of Pennsylvania