1 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns Some Design Patterns Or Best Practices

2 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns Resource Basics

3 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns Some URLs Obvious URLs Collections/Extents –…/staff Returns all staff –…/storesReturn all stores –…/paymentsReturn all payments –…/rentalsReturn all rentals Instances –…/staff/21Return employee 21 –…/stores/11Return store 11 –…/payments/9You get the picture –…/rentals/6You get the picture

4 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns Resource URL and Representation GET …/stores/21 maps to a row in a table with the following columns –Store_id –Manager_staff_id –Address_id –Last_update A straightforward JSON representation is {store_id : 21, manager_staff_id : 11, address_id : 9, last_update : :03:20} There are several issues with this format. Three significant ones are: –How to find the “manager” with id=11 is unclear. I need to know side information not present in the response. –SQL columns names are often “unclear.” I am not exactly sure what “manager_staff” means. –Last_update is a SQL datatype. Understanding how to convert to a corresponding data type in my client application requires knowing the underlying implementation is SQL.

5 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns A Better Representation I would return something like {store: {“data” : {id : 21}, “link” : {“rel” : “self”, “href” : “…/stores/21”}}, {manager : {“data” : {“id” : 11, “name” : {“lastName” : “Ferguson”, “firstName : “Donald”}, {“link” : {“rel” : “storeManager”, “href” : “…/staff/11}}} {address : {“link” : {“rel” : “storeAddress”, “href” : “…/address/9}}, {“timestamp” : “30-Jan-2015”}} Observations –If I got the {…{}…} right in that, I will eat a JSON manual. –I converted the timestamp into something more human/string friendly. –Returning the value of a foreign key columns is wrong. I returned the value and an explicit link to navigate. –I can optionally return some of the data in reference for convenience (columns from the JOIN). –I make the column names a little friendlier.

6 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns Some Additional Observations …/staff/21 maps to a row in a table of the form –Staff_id –First_name –Last_name –Address_id –Picture –… … Some suggestions –Return a name object, not individual fields, i.e. { … …, “name” : {“last” : “Ferguson”, “first” : “Donald”}, … …} –Image is a “BLOB” = “Binary Large Object.” I would return {… …, “Picture” : {“rel”: “picture”, “href” : “…/staff/21/picture”}, … …}

7 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns The BLOB! A GET on –…/staff/21/picture should return –The same kind of thing that you get when you click on – Content-Type: image/jpeg In the Express framework for node.js ( you can set this with the call res.type(‘image/jpeg’) Or just use Amazon S3

8 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns Let ’ s Take a Minute Remember when I talked about REST  “Uniform Interface?” This actually means Resource based –URIs identify (locate) individual resources. –The resources are representations and not the underlying DB schema, file data, … … Manipulation through representations –If I have a representation (may include headers) –I have enough information to reread, modify, delete (subject to security permission) the resource –And do not need “side information,” e.g. knowing that {“manager_id” : 21} needs to become …/staff/21. Self-descriptive: Each “message” contains enough information to process it, e.g. MIME type

9 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns Let ’ s Take a Minute The “pointer” to a resource is the URI The common operations in any data model are (CRUD): –Create = HTTP POST –Retrieve = HTTP GET –Update = HTTP PUT –Delete= HTTP DELETE Datamodels commonly have containment and reference relationships –Containment: Represent logically contained resources as resources, e.g. –Do not return {… …, “lastname” : “Ferguson”, “firstname” : “Donald”, …} –Just because the database represents it that way –When you really mean {“name” : {“last” : “Ferguson”, “first” : “Donald”}} –Reference: Return a typed link to the resource –Relationships are navigable, e.g. –…/store/21/manager returns the resource representation of the correct manager from the “Staff” table –…/staff/32/name/last returns “Ferguson” Datamodels have types, which you represent using MIME types –Application/json  JSON –Application/XML  XML –Image/jpeg  JPG image

10 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns For example, Node.js + Express –Create the URL base at localhost:1234 –Specify the “routes” –app.get(‘URL sub-path’, module.function) –app.put(‘URL sub-path’, module.function) –app.post(‘URL sub-path’, module.function) –app.delete(‘URL sub-path’, module.function) –The JavaScript modules and functions –Perform CRUD on the underlying implementation of the representation, e.g. RDB, MongoDB –And any associated logical like validating data, convert foreign keys to links, etc. There are many others, e.g. Sails, JAX-RS, … This may be Obvious but … How do you implement a service? Use a framework.

11 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns For example, jQuery in a browser And there are many others, e.g. JAX-RS Client API This may be Obvious but … How do you call a service? Use a library.

12 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns Collections

13 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns Collections All datamodels have some form of collections, e.g. –Relational database tables –File system directories –Java.util.Collections.List These map naturally into URIs –…/staff –…/stores –…/addresses But there are a couple of challenges –Locating things in a collection –Paging or iterating through a collection

14 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns Locating Things in a Collection The collection typically has a unique, primary key –With a natural representation in URL/REST, –.../staff/21 –…/stored/36 –Which returns a resource of the form data and links Some collections have non-unique, secondary keys with –A less natural representation in URL/REST, e.g. –.../staff?name.last=“Ferguson”&name.first=“Donald” that –Maps to “SELECT * FROM staff WHERE last_name=“Ferguson” AND first_name=“Donald”; –That is itself also a collection. Some collections support ad hoc, client specified queries –With quasi-natural representation in URL/REST, e.g. –…/addresses?q=“zipcode=12345||state=“California” –Maps to “SELECT * FROM addresses WHERE zipcode=‘12345’ OR state=‘California’ –That is itself also a collection

15 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns Some Observations Why surface secondary keys and not just ad hoc query? –Some databases do not support arbitrary, ad hoc query –DBAs may –Preclude ad hoc queries for various reasons –And only surface predefined, parameterized, compiled template queries –SELECT * FROM CUSTOMERS WHERE zipcode =  …/customers?zipcode=12345 You should –Define your query language in your model –And not just surface the underlying query language, e.g. SQL, MongoDB –Because the underlying language is the implementation, not a representation –And also invites things like SQL injection attacks –Parse, validate and map the query into –Actual implementation language –Actual underlying datamodel, e.g. RDB, MongoDB –And I use q=“….” as the representation of the query string to draw a distinction between –The ability to process ad hoc queries on a collection –Versus a predefined, fixed set of combinations –…/customers?zipcode=12345 is correct –…/customers?zipcode<=12345 is not correct –…/customers?zipcode=12345&type=Gold is not correct

16 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns Iteration/Pagination – The Concept …/earth/animals/humans will return about 6 billion resources And I doubt very much that –The client really wants all 6 billion in one call –Or that the server really wants to return all six billion Which is where iterators and pagination occur –Relational database –DECLARE human_cursor FOR SELECT * FROM humans; –OPEN human_cursor; –FETCH human_cursor INTO …; –Java collections –java.util.Collections.List allHumans; –java.util.Collections.Iterator I = allHumans.getIterator(); –Human h = i.next(); –And other collections/databases have similar concepts

17 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns How do you Represent this in REST? Well, There are a Few Ways Limit and Offset –GET …/earth/humans?offset=0&limit=10 –GET …/earth/humans?offset=10&limit=10 –And this has a natural rendering in some languages, e.g. RDB SELECT * FROM humans LIMIT 10 OFFSET 0; The collection is ordered by a key –GET …/earth/humans?zipcode<=0&limit=10 –Look at the 10 th entry in the result set, which has zipcode=“20135” –And the next 10 are GET …/earth/humans?zipcode>”20135”&limit=10 And you can combine all of this in freaky ways –GET …/earth/humans?q=“hairColor=red&iq>100”&s=“weight,shoesize”&limit=10&offset=10 –Maps to a freaky SQL statement of the form SQL SELECT * FROM humans WHERE hair_color=red AND iq>100 ORDERBY weight, shoe_size OFFSET 10 LIMIT 10

18 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns This Pushes a lot on the Client A well-defined REST API hides this complexity using –RFC 5988 Link headers –The GET returns an HTTP header of the form –Links: ; rel=“next”, ; rel=“prev”, ; rel=“first”, … –rel=“self” is the URL to the page you are on. –rel=“next” is the URL to the “next” page –rel=“prev” is the URL to the “previous” page –rel=“first” is the URL to the “first” page –rel=“last” is the URL to the “last” page –GET on the URL with rel=“next” GETs the next page with a new set of next, prev, first, … related URLs This enables –The client to specify a limit or the server to specifyc a limit –And enable the client to iterate without know how the server is implementing iteration, e.g. –LIMIT and OFFSET –ORDER BY and values –… …

19 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns If You Do Not Like Headers GET …/staff?status=active&limit=2 returns something like { –“data” : –[ –{“id” : 21, “manager” : {“link” : { “rel” : “manager”, “href” : “…/managers/9”}}}, –{“id” : 11, “manager” : {“link” : { “rel” : “manager”, “href” : “…/managers/101”}}}} –] – “links” : –[ –{“rel” : “next”, “href” : “…/staff?status=active&offset=2&limit=2}, –{“rel” : “first”, “href” : “…/staff?status=active&offset=0&limit=2}, –{“rel” : “last”, “href” : “…/staff?status=active&offset=210934&limit=2} –] } Why did I highlight this?

20 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 2: REST Details and Patterns A subtlety A GET on –…/humans?iq=“100” –…/humans?q=“id=100&&hairColor=red” Implicitly –Creates/computes a new collection resource “just for you” –That is the result of running the query or using the keys But, –REST is stateless and the server is not going to “keep the result” for “just for you” –So you have to tell the server how to re-compute the result But, –You may not be the only one CRUD-ing humans –So freaky stuff can happen. For example, following the “self” link might not return the same JSON object that you just got. The freaky stuff can happen in any database. This is why RDBs have isolation levels for cursors. We cannot use isolation levels –Not really compliant with REST –Does not suit “Web scale” and “loosely coupled” applications –Which means we have to handle in other ways