1. IRMIS3 Data Service and Application Layer Gabriele Carcassi Oct 14 2008

2. Disclaimer The specifications are not final – parts are missing and there will be a certain amount of evolution on the part that exist – good time to participate The overall architecture and the guidelines, though, are stable – we will concentrate on those

3. IRMIS3 architecture IRMIS3 RDB (MySQL) IRMIS3 Data service Java applets AJAX components Scripts and CLI (perl, Python, …) Browser (Firefox, IE, …) XML protocol Web server (Glassfish)

4. Data service The architecture is your typical Service Oriented Architecture (SOA) and service is a Representational State Transfer Web Service (REST-WS) Allows to capture the business logic (all the rules that separate valid transactions from invalid transactions) in one place – Clients can be “stupid” Exposes the data in a well defined xml format – Allows the internals of the database to change – Allows to use any available xml tools

5. Data service: guidelines The service is responsible that all data modifications are valid and to always return valid information – It should fail to run invalid changes (i.e. installing a building in a room) – If it doesn’t, it’s a bug on the server – Note that valid != correct Service is stateless

6. XML protocol (REST style WS) IRMIS3 software stack Few database utilities: backup, consistency check, etc… Client Server Integration with external tools (i.e. physcs) 3 rd party Perl/Pyton scripts 3 rd party Java apps Jython scripts Web applications JavaScript bridge Applets and Widgets Java Client API Data Service layer Database layer

7. How you can interact with IRMIS3 Just use the GUI Run scripts that others develop Read the XML directly Write phyton/perl scripts that consume the XML Write Jython script using the API Develop your own GUI

8. XML protocol Few database utilities: backup, consistency check, etc… Client Server 3 rd party Java apps Jython scripts Web applications JavaScript bridge Applets and Widgets Java Client API Data Service layer Database layer

9. XML protocol An XML schema defines the elements and attribute used by the service. To read: each different query corresponds to a different web address. HTML parameters are used for query parameters. To write: send an XML which represent a transaction. Each element in the transaction is processed independently.

10. XML protocol: an Oscilloscope......

11. XML protocol: a transaction...

12. XML protocol guidelines Queries are designed to return “chunks” with related information – minimize roundtrips at the cost of repetition a componentType returns all the details of the ports, interfaces, manufacturers, etc… port information for a component returns all the ports, pins, portType, cables and conductors – more or less the opposite of a database query Transactions are executed in a single DB transaction No guarantee that the data you read wasn’t modified when you write (no “FOR UPDATE”) – Will provide the timestamp for the last modification

13. Testing the protocol

14. Demo Let’s have a look at the Data Service

15. Exercise Using the test interface – List the manufacturers (at /IRMISDataService/data/manufacturers) – Create a new manufacturers called “ACME” (schema documentation at /IRMISDataService) – List the manufacturers again – Delete manufacturer “ACME”

16. XML protocol (REST style WS) IRMIS software stack Few database utilities: backup, consistency check, etc… Client Server Integration with external tools (i.e. physcs) 3 rd party Perl/Pyton scripts Web applications JavaScript bridge Applets and Widgets Data Service layer Database layer

17. Java API: design principles The Java API implement the XML protocol and provides objects that represent the data All reads are cached: no need to keep caches in your code. All reads are synchronized – you can have reads on multiple threads on the same data and will not trigger multiple reads on the server (extremely useful for GUIs)

18. Java API design principles The read access is object oriented – Data is exposed through interfaces Component, ComponentType, Manufacturer – Some classes function as “finders” for the classes containing the data Components.getById(123), FormFactors.getByName(“Rack”) – You can follow references from one object to the other component.getComponentType().getManufacturer()

19. Java API design principles The write access is protocol oriented – You create a transaction – You call methods to add elements in the transaction Manufacturers.createManufacturer(transaction, “ACME”) – You save the transaction, which may fail – The read access is refreshed only after the data is saved

20. Importing components from Excel (using Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful"

21. Importing from Excel (Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful" Load excel file

22. Importing from Excel (Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful" Connect to the service

23. Importing from Excel (Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful" Loop over the rows

24. Importing from Excel (Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful" Get data out Create components

25. Demo Let’s browse the API documentation

26. Exercise Write a Java (or Jython, JRuby, Groovy, …) script that exports the components (id, Type name, Serial, FieldName) to an Excel file – API and documentation at /IRMISDataService – Use JXL from http://jexcelapi.sourceforge.net/

27. XML protocol (REST style WS) Applets and javascript Few database utilities: backup, consistency check, etc… Client Server Integration with external tools (i.e. physcs) 3 rd party Perl/Pyton scripts 3 rd party Java apps Jython scripts Java Client API Data Service layer Database layer

28. Why applets?!? Especially given the bad history of applets, and the number of Web 2.0 javascript framework (jMaki, GWT, DOJO, DWR, …)? The code generated by these framework cannot be directly modified: you have to know the framework By providing you a Java API and applets, you can more easily develop on top: either by writing a couple of lines of javascript, or by integrating in a javascript framework

29. What we provide A set of Swing models to access the data – These models will be able to populate themselves asynchronously (some already can), so that no logic needs to be put in the GUI A set of Swing components – You are free to take these components and put them in your application A way to wrap these components in applets – So you can script on top of them with web pages

30. Scripting applets We had a summer student (Vikram Aravamudhan) write a general purpose library that allows to register javascript to Swing events

31. Scripting applets registerEvent(); function componentTypeChanged(res1) { document.getElementById('manufacturer').value = res1.getSource().getSelectedValue().getManufacturer().toString(); document.getElementById('description').value = res1.getSource().getSelectedValue().getDescription().toString(); document.getElementById('ffactor').value = res1.getSource().getSelectedValue().getFormFactor().toString(); document.getElementById('name').value = res1.getSource().getSelectedValue().getName(); } function registerEvent() { // The applet might not be ready when this function is called the first time. // If it fails, it retries after a second. try { document.applet1.registerEvent("valueChanged","componentTypeChanged"); } catch (ex) { setTimeout('registerEvent()', 1000); } The idea is that each IRMIS applet wraps around a JComponent, and you can register to the events that that component exposes.

32. Demo Let’s have a look at the documentation

33. Exercise Run the scripting example Modify the example to use the ComponentTreeApplet – Applet documentation at /IRMIS/ – Glassfish unpacks the application in glassfish/domains/domain1/applications/j2ee- modules/IRMISWebUI (This is a hack to get stuff done quickly here)