© Donald F. Ferguson, 2015. All rights reserved. Topics in Computer Science: COMS E6998-07-2015-03 Micro-service Application and API Development Lecture.

© Donald F. Ferguson, 2015. All rights reserved. Topics in Computer Science: COMS E6998-07-2015-03 Micro-service Application and API Development Lecture 1: Overview, Concepts, REST Dr. Donald F. Ferguson Donald.Ferguson@software.dell.com (Admin: Melissa.Peoples@software.dell.com)

3 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Contents Overview15 min. –A little bit about me. –Initial lecture schedule. –Coursework and grading. Concepts35 min. –Evolution to micro-services. –Micro-services concepts. –APIs. Datamodel –Basic concepts –Traditional realization –Transactions, REST –Concepts and overview. –Some best practices. 1 st Project and related concepts Q&A, discussion10 min.

5 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Monolithic to Micro Cart Functions Java SQLite Recommendation Functions Node.js Redis Catalog Functions PDP MongoDB XXX MMM NNN Content Functions Ruby Amazon S3

6 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST SOA vs Microservices http://www.pwc.com/us/en/technology-forecast/2014/cloud- computing/features/microservices.jhtml

7 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Inject application implementation into reusable SW containers Reusable infrastructure containers Reusable SW containers but with core technology and frameworks

9 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST API Economy Sometime you just reuse an API instead of building a micro-service.

10 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST The Course – We Will Cover Patterns, technology and best practices for –Implementing base microservices. –Assembling microservices into “solutions.” –Achieving scalability, availability, agility, etc. Web callable infrastructure services that accelerate implementing microservices, e.g. –Databases –Messaging –… … Web callable application/business services that accelerate implementing a microservise’s business/domain functions.

11 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Microservice Patterns New Microservice PatternsClassic Patterns, New Realization http://microservices.io/patterns/ “Enterprise Integration Patterns,” Fowler et al.

14 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Monolithic to Micro Cart Functions Java SQLite Recommendation Functions Node.js Redis Catalog Functions PDP MongoDB XXX MMM NNN Content Functions Ruby Amazon S3

15 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Database Model are Complex, even examples and samples, e.g. MySql Sakila Sample Database

16 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Some Concepts Foreign key relationships –Represent 1-1, 1-N relationships –Have “behavior,” e.g. –On Delete would prevent deleting a country if there is a city whose country_id is the country’s id. –On Cascade would automatically update all city.country_id when country.id changes Defining indices is important to avoid –Scanning the entire city table to –To find cities in a given country I would not put strings in a table for most words and string –Putting “Spain” for a country name –Prevents localization and national language enablement –Use symbols into localization resource bundles

18 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST What can we Learn? The implementation of the SOA contract/REST interface –Is a set of verbs on URLs –That manipulate a logical data model. Every logical data model has a common set of concepts that materialize through REST –ID  URI/URL –Collections supporting –Primary key  …/Customers/21 –Non-unique, secondary keys  …/Customers/Zipcode/12345 –Ad hoc query (SELECT WHERE (… …))  …/Customers?q=“id<=50&lastname=Ferguson” –Projection –SELECT iq, lastname FROM Customers  …/Customers?”Fields=iq,lastname” –UPDATE iq, shoessize WHERE …  PUT {{iq, “50},{…}}  …/Customers –Foreign keys/join tables  Hyperlinks –Iterators –SELECT * FROM Customers CREATE Cursor …  –GET …/Customers?Offset=40&Pagesize=20 –Thread/callback/promise  Asynchronous REST responses –Metadata/reflection: SQL DESCRIBE TABLE  Web UI for driving the REST API –Stored procedures  PUT…/Commands/… –Events/Notifcations  Feeds

20 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST “Decomposing applications for deployability and scalability,” Chris Richardson, http://plainoldobjects.com/

21 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Simple to manage Atomicity Consistency Isolation Durability

23 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST ACID Transactions ACID transactions –Atomic: A set of writes all occur or none occur. –Durable: The data “does not disappear,” e.g. write to disk. –Consistent: My applications move the database between consistent states, e.g. the transfer function works correctly. Isolation –Determines what happens when two or more threads are manipulating the data at the same time. –And is defined relative to where cursors are and what they have touched. –Because the cursor movement determines what you are reading or have read.

24 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Monolithic to Micro Cart Functions Java SQLite Recommendation Functions Node.js Redis Catalog Functions PDP MongoDB XXX MMM NNN Content Functions Ruby Amazon S3 ACID is pretty straightforward When there is a logical, single DB. With integrated code moving from consistent state to consistent state. But, this gets really hard when Data is federated across multiple, independent microservices and DBs. With different approaches to transactions, replication, scalability, … …

25 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST CAP Theorem The CAP theorem, also known as Brewer's theorem, states that it is impossible for a distributed computer system to simultaneously provide all three of the following guarantees: Consistency (all nodes see the same data at the same time) Availability (a guarantee that every request receives a response about whether it succeeded or failed) Partition tolerance (the system continues to operate despite arbitrary partitioning due to network failures)

27 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Representational State Transfer (REST) People confuse –Various forms of RPC/messaging over HTTP –With REST REST has six core tenets –Client/server –Stateless –Caching –Uniform Interface –Layered System –Code on Demand

28 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST REST Tenets Client/Server (Obvious) Stateless is a bit confusing –The server/service maintains resource state, e.g. Customer and Agent info. –The conversation is stateless. The client provides all conversation state needed for an API invocation. For example, –customerCursor.next(10) requires the server to remember the client’s position in the iteration through the set. –A stateless call is customerCollection.next(“Bob”, 10). Basically, the client passes the cursor position to the server. Caching –The web has significant caching (in browser, CDNs, …) –The resource provider must –Consider caching policies in application design. –Explicitly set control fields to tell clients and intermediaries what to cache/when.

29 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST REST Tenets Uniform Interface –Identify/locate resources using URIs/URLs –A fixed set of “methods” on resources –myResource.deposit(21.13) is not allowed –The calls are –Get –Post –Put –Delete –Self-defining MIME types (Text, JSON, XML, …) –Default web application for using the API –URL/URI for relationship/association Layered System: Client cannot tell if connected to the server or an intermediary performing value added functions, e.g. –Load balancing –Security –Idempotency Code on Demand (optional): Resource Get can deliver helper code, e.g. –JavaScript –Applets

31 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Anatomy of a URL SSOL for the Classlist https://ssol.columbia.edu/cgi- bin/ssol/DhbtiwfsFOMOeFQaDwqxAh/?p%.5Fr%.5Fid=k0F2vZ4ccAhzbcAg0Ql K4h&p%.5Ft%.5Fid=1&tran%.5B1%.5D%.5Fentry=student&tran%.5B1%.5D%. 5Fterm%.5Fid=20143&tran%.5B1%.5D%.5Fcid=COMSE6998&tran%.5B1%.5 D%.5Fsecid=005&tran%.5B1%.5D%.5Fsch=&tran%.5B1%.5D%.5Fdpt=&tran %.5B1%.5D%.5Fback=&tran%.5B1%.5D%.5Ftran%.5Fname=scrs This is –Not REST –This is some form of Hogwarts spell –This is even bad for a web page

32 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Anatomy of a URL HTTP://www.somedomain.edu/...The “server” containerHTTP://www.somedomain.edu/ …/ssol/…The module/component …/listManagerThe Application Object or …/Class/COMSE6998-01Entity Class (“Extent”) and ID.../WaitingList/…Contained Resource GET, POST, … on URLfor CRUD Some details –…/WaitlingList/dff9/IQPath navigation into resources –…/WaitlingList?op=“Approve”?CUID=“dff9”Method

34 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Implementation Observations Define a collection /QueuedResponses –A client can call …/QueuedResponses/21 to get a specific response. –You already know how to do this for …/Customer –The data format in the table is {id, status, JSONString} A simple implementation would be writing a façade –Accept request –Create new table entry with status = “in progress” –Return 202 and URL –Call the actual implementation –Update the database table entry with the JSON result Most application platforms have middleware approaches to support registering callbacks, threads, etc. The implementation would typically –Invoke some long running action, e.g. DB query, workflow process and register a callback –The callback implementation updates the entry in the response table.

35 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Ad Hoc Query Every collection should support ?q=“… …” –…/Customers?q=“lastName=21&IQ<21” –q is a string encoding a set of triplets with elements –Resource field, e.g. “lastName” –Comparison operation, e.g. “=“, “>”, … –Comparison value. Your code needs to –Parse and validate the query string. –Rewrite the string in the query language of the underlying database, e.g. Where clause in SQL –Execute the query –Refine the result set if the underlying database does not support query capabilities that you are surfacing through your API.

36 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Pagination {“data”: [{“user_id”:”42”, “name”:”Bob”, “links”:[{“rel”:”self”, “href”:”http://api.example.com/users/42”}]}, {“user_id”:”22”, “name”:”Frank”, “links”: [{“rel”:”self”, “href”:”http://api.example.com/users/22”}]}, {“user_id”:”125”, “name”: “Sally”, “links”:[{“rel”:”self”, “href”:”http://api.example.com/users/125”}]}], “links”: [{“rel”:“first”, “href”:”http://api.example.com/users?offset=0&limit=3”}, {“rel”:“last”, “href”:”http://api.example.com/users?offset=55&limit=3”}, {“rel”:“previous”, “href”:”http://api.example.com/users?offset=3&limit=3”}, {“rel”:”next”, “href”:”http://api.example.com/users?offset=9&limit=3”}]}

37 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Implementation Considerations Query rewrite –…/Customers?q=“lastname=Ferguson&id<5”&limit=10&offset=5 –Neatly translates into an SQL statement –Select * from customers where … limit=5 offset=5 Other databases have similar concepts. You may have to –Rewrite a push the query down –Build a result cache in another store that supports limit/offset –Paginate through the cache You should also consider adding –“field=lastname,IQ,color” –To enable selecting a subset of fields

39 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Example Considerations Service Endpoint –Endpoint –Endpoint Encryption Requests and Authentication –Request Headers –Request Timestamps –Request Authentication –Response Headers Resources –Resource Requests –Resource Representation –Resource Methods –Synchronous Operations –Asynchronous Operations –Success Response Codes –Failure Response Codes Resource Data Types –Atomic Types –Complex Type - Object or Structure –Resource Relationships –Resource References Pagination –Through HTTP Link Header –Syntax and Example of Pagination Link Header –Consistency Across Page Requests Versioning –Version Header –Version URI Saying “REST is not enough You have to define a set of patterns/ conventions of URLs, headers, …

40 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Endpoint /a/b/c?x=7&y=21 is a pretty straightforward concept, but … –How do I get info about customer “Ferguson?” –…/Customer/Ferguson/Donald –…/Customer?lastName=“Ferguson”&firstName=“Donald” –??? –Do I really want to –Find info about Don using …/Customer/Ferguson/Donald –Find info about agent using …/Agents?id=“21” –How does it work if I can find customer by name or phone number? –How do I set a relationship between customer and agent? –PUT …/Relationship/AgentFor?agent=“21”&”Customer=“Ferguson” –Or two PUTS, one on Customer and one on Agent? In the same way you have to define a framework for your application, you have to define a shape/pattern in your REST API model.

41 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Headers – Some Choices I Made HTTP Request HeaderValueMandatory auth-timestamp:The current POSIX time.Yes auth-key: The user or client’s unique API KEY. Yes auth-signature: The HMAC-SHA256 digest for the request. Yes api-version:(Optional) API version stringNo Accept: (Optional) application/xml or application/json No Nonce:One time UUID to enable idempotency/duplicate detection

42 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Response Codes OperationHTTP RequestHTTP Response Codes Supported READGET 200 - OK with message body 204 - OK no message body 206 - OK with partial message body CREATEPOST 201 - Resource created (Operation Complete) 202 - Resource accepted (Operation Pending) UPDATEPUT 202 - Accepted (Operation Pending) 204 - Success (Operation Complete) DELETE 202 - Accepted (Operation Pending) 204 - Success (Operation Complete) Examples of Link Headers in HTTP response: Link: ;rel=monitor;title="update profile" Link: ;rel=summary;title=”access report” 202 means Your request went asynch. The HTTP header Link is where to poll for rsp.

43 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Failure Response Code ErrorResponse Code Invalid Parameter400 - Invalid parameter Authentication401 - Authentication failure Permission Denied403 - Permission denied Not Found404 - Resource not found Invalid Request Method405 - Invalid request method Internal Server Error500 - Internal Server Error Service Unavailable503 - Service Unavailable

44 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Design Pattern Choices Resource Requests Collectionhttps://ENDPOINT/NAMESPACE/RESOURCE[?QUERY_PARAMETERS] Resourcehttps://ENDPOINT/NAMESPACE/RESOURCE/RESOURCE_ID[?QUERY_PARAMETERS] Collection OperationHTTP Request Get all items in the collection GET /collection Should also return the URI of the collection itself. Get an particular item in the collectionGET /collection/itemId Get items match certain criteriaGET /collection?property1=’value’ Add a new item to the collection POST /collection contents of new item … Get items starting at 100 with page size=25 GET /collection?start=100&pageSize=25 Support for Map Array Collection

45 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Relationships {“membership”: { “URI” : “http://dell.com/memberships/m12356”,http://dell.com/memberships/m12356 “created” : “2013-08-01T12:00:00.0Z”, “owner” : “user123456”, “expire” : “never”, “group” : { “ref” : “http://dell.com/groups/g123456” },http://dell.com/groups/g123456 “server” : { “ref” : “http://dell.com/servers/s123456”}http://dell.com/servers/s123456 } "link": { "href": "http://dell.com/api/resource1",http://dell.com/api/resource1 "rel": "self", “title” : “server-s123456” } Relationship as a resource Relationship as a field in resource

46 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Navigating Through Result Set GET on collections, maps, … needs pagination and cursors –Limit: What is the maximum number of elements you want? –QueryID: A tag for the query that produced the original result set –Offset references a specific element in a “page.” There is a standard for linking resources in logical sets, e.g. –Link: ; rel="previous"; title="previous chapter“> –Indicates that "chapter2" is previous to this resource in a logical navigation path. Your API/framework can use this for result sets –Example 1 –GET /api/customers?status=“Gold” returns some number of “Gold” customers and –Link Header for “next page” is Link: ; rel="next last“ –Which is the URL for the “cursor.next set,” which has 50 elements and is also “last” –Example 2 –Get returns the “next” from example 1 –With Link Link: ; rel="prev first" –Allowing you to go backwards to the previous “page.”

49 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST SOA Component Architecture  Microservices Five elements still matter 1.Interface and binding.

54 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST X-Axis and Y-Axis Scale-Out through Shared database Cloned, stateless code

56 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST 1 st Projects –Simple content management –Mail verification for registration; CAPTCH

58 © Donald F. Ferguson, 2015. All rights reserved.Micro-service Application and API Development (E6998-07-2015-03) – Lecture 1: Overview, Concepts, REST Scaling https://devcentral.f5.com/articles/the-art-of-scale-microservices-the-scale-cube-and-load-balancing

© Donald F. Ferguson, 2015. All rights reserved. Topics in Computer Science: COMS E6998-07-2015-03 Micro-service Application and API Development Lecture.

Similar presentations

Presentation on theme: "© Donald F. Ferguson, 2015. All rights reserved. Topics in Computer Science: COMS E6998-07-2015-03 Micro-service Application and API Development Lecture."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

© Donald F. Ferguson, 2015. All rights reserved. Topics in Computer Science: COMS E6998-07-2015-03 Micro-service Application and API Development Lecture.

Similar presentations

Presentation on theme: "© Donald F. Ferguson, 2015. All rights reserved. Topics in Computer Science: COMS E6998-07-2015-03 Micro-service Application and API Development Lecture."— Presentation transcript:

Similar presentations

About project

Feedback