1. X2R Spec 1

2. Change log DateVersionPeopleNote 2013/11/01V0.0.1Chien-Wei Yu, Anderson Ou First draft, add X2R files spec. 2013/12/16V0.0.2Anderson Ou, Doc YangAdd X2R Development. 2014/04/10V0.0.3Anderson Ou, Doc Yang, Gentry Yao Code review 1 st time 2014/04/25V0.0.4Anderson Ou, Doc Yang, Gentry Yao Code review 2 nd time 2014/07/30V0.0.5Doc Yang, Gentry YaoCode review 3 rd time 2014/08/06V0.0.6Doc Yang, Gentry YaoCode review 4th time 2

3. Outline VR X2R Service Overview VR X2R Use Spec VR X2R Development Environment Coding Standard Use Scenario Input / Output of components Source Code Files Structures 3

4. Services Overview Software components: X2R – Replace the URI in the translated RDF with “live and connected” URI USS – URI search service UMS – URI management service Extractor/Mapper – Pre/post-processor for handling URI in the RDF Services: Web-based GUI (X2R, USS and UMS) Web-based API (X2R, USS, UMS and Extractor/Mapper) Operation mode Interactive mode Batch mode 4

5. Use Spec GUI on browser Recommended resolution: 1280*768 Browsers: Google Chrome (main development platform) Mozilla FireFox Internet Explorer 11 or later API Services included: X2R, USS, UMS and Extractor/Mapper Web-based API with HTTP standard methods GET and POST 5

6. Development Environment Programming Languages: Backend: PHP Frontend: JavaScript, HTML, CSS Others: JSON, XML, AJAX (jQuery AJAX) Libraries: Backend: FirePHP for debugging, cURL for HTTP operations Frontend: jQuery, jQuery UI, Twitter Bootstrap v3, Syntax highlighter, DataTables Servers: Machine: Linux (CentOS), Apache server Database: Apache MySQL, OpenLink Virtuoso 6

7. Development Environment Browser Compatibility (Ref)Ref Google Chrome (main development platform) Mozilla FireFox Internet Explorer 11 or later Suggested Screen Resolution (Ref)Ref W 1366 X H 768 We make sure webpages do not need to be scrolled horizontally in width 1366 pixel. 7

8. Development Tools Editor: Sublime Text 3, Netbeans Code Sniffer: PHP_CodeSnifferPHP_CodeSniffer Documentor: (PHPDocumentor, Doxygen, etc. to be confirmed) Unit testing: PHPUnitPHPUnit Version Control: Git on OpenISDM GitlabOpenISDM Gitlab Issue tracker: OpenISDM GitlabOpenISDM Gitlab 8

9. VR X2R Development Programming Languages: Backend: PHP Frontend: JavaScript, HTML Others: JSON, XML, AJAX Libraries: Backend: Easy RDF (http://www.easyrdf.org/support)http://www.easyrdf.org/support Frontend: jQuery, jQuery UI, Twitter Bootstrap v3 Servers: Machine: Linux (CentOS), Apache server Database: OpenLink Virtuoso, Apache MySQL 9

10. Coding Standard PHP- PHP Framework Interop Group PSR-0 Autoloading standard PSR-1 Basic coding standard PSR-2 Coding style guide Extra rules. See Appendix. JavaScript - Google JavaScript Style Guide See next pages for URL 10

11. Comment style Put the description at the beginning of each file. 11

12. Comment style Put function description at the beginning of each funciton 12

13. Comment style Put comment before code block. Add a blank comment line before and after the comment block. 13

14. Comment style Put the TODO note in the wherever needed. 14

15. Code Sniffer Run PHP Code Sniffer to check the coding style of files in the project every time before you push to the remote branch. Run following script on the Windows command line. >> phpcs --standard=PSR2.\ Make sure there is no error or warning show up the test report. Run following script on the Windows command line. >> phpcs --standard=PSR2 --report=summary.\ 15

16. Reference for Coding Standard PHP Framework Interop Group Front End Coding Style (Blog article)Front End Coding Style Google JavaScript Style Guide, http://google-styleguide.googlecode.com/svn/trunk/javascriptguide.xml Mozilla Webdev Bootcamp 16

17. USS TranslatorUserExtractorUMSMapper Scenario: Translator first, USS/UMS next Send XML Check format Translate input, create temp URIs Ask X2R Return temp RDF Get temp RDF Send temp RDF Return File-Term mapping Extracting terms Create URI Return URI Check or create URIs Search URIs Send term to search URI Send result X2R UI Map and replaced URIs Ask do terms/URIs mapping Return RDF with replaced URI Send File-Term-URI mapping Return final RDF RDF translation Step1: Extract terms Step2: Request USS and UMS Step3: Map terms /URI

18. Input files ExtractorX2R GUIUSSMapper Output files RDF Input files Translator ExtractorX2R GUIUSSMapper Output files RDF XML, CSV, JSON File-Term mapping File-Term-URI mapping RDF RDF with replaced URI File-Term mapping RDF File-Term-URI mapping RDF with replaced URI Need translation: XML, CSV, JSONNo need translation 18 UMS Input / Output of components

19. Input and Output of Components System Input files: XML, JSON, CSV, RDF Translator: Input files: XML, CSV(tbc), JSON(tbc) Output file: RDF Extractor: Input files: RDF (original / translated RDF), XML (tbc), CSV (tbc), JSON (tbc) Output file: File-Term mapping X2R: Input files: File-Term mapping (Required. Terms only are acceptable), Original files (Optional. For carry to mapper) Output file: File-Term-URI mapping Mapper: Input files: File-Term-URI mapping and Original files Output file: Final RDF with replaced URIs 19

20. Class of USS class sparqlRepositoryInformation class sparqlRepositoryOperationStatus class initialTask class uriSearchTask

21. class sparqlRepositoryInformation $codeName $dataSourceName $siteURL $sparqlEndpointURL $operationStatus

22. class sparqlRepositoryOperationStatus public GetRepositoryOperationalStatus() - get the information of endpoint private setRepositoryOperationalStatus() - set the information of endpoint

23. class initialTask public SynthesisSingleSparqlQuery () - set the parameter of sparql query public SynthesisMultipleSparqlQueries() - set the parameter of the set of tasks pravite composeSparqlQuery() - compose query according to endpoints public GetSearchedResult() - get the result from the sparql query private setSearchedResult() - set the result from the sparql query

24. class uriSearchTask public SearchSingleRepository() - send the sparql query to endpoint public SearchMultipleRepositories() - send the sparql query to endpoints of the set of tasks public integrateSearchedResults() - integrate the result from different search tasks public SortIntegratedSearchedResults() - sort the result ordered by different rules public GetIntegratedSearchedResults() - get the integrated results

25. X2R code structure

26. X2RConverterGUI x2rConverterIndex GetQueryTerms uriTail tokenize Extractor USS Structure queryTermSelection uriSelection UMS Mapper SerializeRdf Refactoring refinedRdfFileGeneration GetIntegratedSearchedResults SearchMultipleRepositories SynthesiseMultipleSparqlQueries GetRepositoryOperationalStatus

27. X2RConverterGUI x2rConverterIndex GetQueryTerms uriTail tokenize Extractor USS Structure queryTermSelection uriSelection UMS Mapper SerializeRdf Refactoring refinedRdfFileGeneration $baseURL = '/EM/extractor'; $params = array( 'excludedNamespaces' => '', 'checkUrisStatus' => '1', 'rdfContent' => $statement, ); $querypart = '?'; foreach ($params as $name => $value) { $querypart = $querypart. $name. '='. urlencode($value). '&';} $extractorURL = $baseURL. $querypart; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $extractorURL); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLINFO_HEADER_OUT, true); $response = curl_exec($ch); $extractionResult = json_decode($response); GetIntegratedSearchedResults SearchMultipleRepositories SynthesiseMultipleSparqlQueries GetRepositoryOperationalStatus

28. X2RConverterGUI x2rConverterIndex GetQueryTerms uriTail tokenize Extractor USS Structure queryTermSelection uriSelection UMS Mapper SerializeRdf Refactoring refinedRdfFileGeneration window.onload = function(){ console.log("loading ready..."); $("#btn-submit").click(function(){ // Clear table body each time $('table').empty(); // Get input parameters from user var q = $('input[name="q"]').val(); var output = $('#output').val(); $.getJSON( "../USS/USSmain.php", { "q": q, "output": "application/json" }).done(function(data){ //ToDo: Error handling } GetIntegratedSearchedResults SearchMultipleRepositories SynthesiseMultipleSparqlQueries GetRepositoryOperationalStatus

29. X2RConverterGUI x2rConverterIndex GetQueryTerms uriTail tokenize Extractor USS Structure queryTermSelection refinedRdfFileGeneration UMS Mapper SerializeRdf Refactoring uriSelection window.onload = function(){ console.log("loading ready..."); $("#btn-submit").click(function(){ // Clear table body each time $('table').empty(); // Get input parameters from user var p = $('input[name="q"]').val(); var output = $('#format').val(); $.getJSON( "../EM/mapper.php", { "mapping": p, "format": "application/json" }).done(function(data){ //ToDo: Error handling } GetIntegratedSearchedResults SearchMultipleRepositories SynthesiseMultipleSparqlQueries GetRepositoryOperationalStatus

30. Thank you very much!! 30

31. Source Code Files Structures X2RConverterGUI x2rConverterIndex.phpthe index page of x2r queryTermSelection.phpverify / select the extracted terms uriSelection.php search / select the corresponding URI of the term refinedRdfFileGeneration.phpfinalize the X2R process and download the results Database operation (Create/Read/Update/Delete) for X2R dbCrud.phpThe DB operations with API support dbCrudClass.phpThe class file of DB operation object dbCrudExample.phpThe example to demo CRUD the database USS 31

32. Class of USS class sparqlRepositoryInformation class sparqlRepositoryOperationStatus class singleUriSearchTask class multipleUriSearchTasks

33. class sparqlRepositoryOperationStatus public GetRepositoryOperationalStatus() - get the information of endpoint private setRepositoryOperationalStatus() - set the information of endpoint

34. class singleUriSearchTask public SynthesisSingleSparqlQuery () - set the parameter of sparql query pravite composeSparqlQuery() - compose query according to endpoints public SearchSingleRepository() - send the sparql query to endpoint public GetSearchedResult() - get the result from the sparql query private setSearchedResult() - set the result from the sparql query

35. class multipleUriSearchTasks public SynthesisMultipleSparqlQueries() - set the parameter of the set of tasks public SearchMultipleRepositories() - send the sparql query to endpoints of the set of tasks public integrateSearchedResults() - integrate the result from different search tasks public SortIntegratedSearchedResults() - sort the result ordered by different rules public GetIntegratedSearchedResults() - get the integrated results

36. Source Code Files Structures X2R x2r.phpThe index page of x2r x2rStep1.phpStep1 is to verify / select the extracted terms x2rStep2.phpStep2 is to search / select the corresponding URI of the term x2rStep3.phpStep3 is to finalize the X2R process and download the results Database operation (Create/Read/Update/Delete) for X2R dbCrud.phpThe DB operations with API support dbCrudClass.phpThe class file of DB operation object dbCrudExample.phpThe example to demo CRUD the database USS index.htmlThe front page of USS search.phpThe main file of USS with API support sparqlTaskClass.phpThe class file of search task object 36

37. Source Code Files Structures X2RConverterGUI x2rConverterIndex.phpThe index page of x2r selectSearchedTerm.phpStep1 is to verify / select the extracted terms selectUriTerm.phpStep2 is to search / select the corresponding URI of the term finalizeProcessResult.phpStep3 is to finalize the X2R process and download the results Database operation (Create/Read/Update/Delete) for X2R dbCrud.phpThe DB operations with API support dbCrudClass.phpThe class file of DB operation object dbCrudExample.phpThe example to demo CRUD the database USS 37

38. Class of USS class SparqlEndpointInformation class SparqlEndpointOperationStatus class SearchTask class SearchTasks

39. class SparqlEndpointOperationStatus $sparqlEndpoint = new SparqlEndpointInformation(); public checkEndpointOperationStatus() - check if endpoint alive or not public getEndpointOperationStatus() - get the information of endpoint private setEndpointOperationStatus() - set the information of endpoint

40. class SearchTask public initializeSearchTask() - set the parameter of sparql query public composeSparqlQuery() - compose query according to endpoints public performSearchARepository() - send the sparql query to endpoint public getSearchedResult() - get the result from the sparql query private setSearchedResult() - set the result from the sparql query

41. class SearchTasks public prepareSearchTasks() - set the parameter of the set of tasks public performSearchMultipleRepositories() - send the sparql query to endpoints of the set of tasks public integrateSearchedResults() - integrate the result from different search tasks public sortIntegratedSearchedResults() - sort the result ordered by different rules public getIntegratedSearchedResults() - get the integrated results