1. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 1 Introduction to GRIDs Keith G Jeffery Director, IT & Head, ITD kgj@.rl.ac.uk http://www.itd.clrc.ac.uk/

2. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 2 CLRC-RAL Site

3. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 3 CONTENTS What GRIDs Can Do In the Beginning A Possible Architecture Scenarios Conclusion

4. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 4 What GRIDs Can Do : Science provide (by coupling) massive compute power to solve problems provide relevant information from multiple sources generate new knowledge ==> improve IPR and talent in science / engineering / technology

5. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 5 What GRIDs Can Do : Business Provide an e-System for: –B2C: business to customer with better selectivity and choice, more security –B2B: business to business with greater automation and demand-supply matching –business chains - processes in cooperating businesses interlinked - end-to-end ==> major centre for e-business

6. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 6 What GRIDs Can Do : Health improved understanding of how illness happens relationship of illness to genome relationship of illness to environment and social factors ==> healthier population

7. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 7 What GRIDs Can Do : Lifelong Learning distance learning with modules supplied from appropriate sources e-libraries, courseware - hypermedia remote experiment control - explore simulations and visualisation - insight virtual reality for training (like flight simulators) ==> human capital improved

8. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 8 What GRIDs Can Do : Environment much greater availability of data much greater compute power for modelling and display of results ability to work with complex multi- dimensional, multivariate data ==> understanding the changing world ==> managing environmental disasters

9. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 9 What GRIDs Can Do : Culture e-art galleries and e-museums e-performances all with good annotation and cross-links to reference material ==> generally available in-depth appreciation

10. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 10 CONTENTS What GRIDs Can Do In the Beginning A Possible Architecture Scenarios Conclusion

11. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 11 IN THE BEGINNING… GRIDs Idea GRIDs Architecture GRIDs Business Model Computation/Data Information Knowledge

12. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 12 The GRIDs Idea Need for Strategic Plan for IT to support UK SET base Initial paper –grand challenge science projects (vertical) –unifying middleware and metadata (horizontal) linking resources –involve industry for technology transfer –a 3-layer GRIDs

13. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 13 The GRID Bible

14. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 14 The Big Idea: What it Provides User Appliance ‘The Wall’ The GRIDs Environment Plug-in PC Palmtop Mobile.. Personal Communication Personal Shopping Hobbies, family activities Business Communication Business Dealing Business Information

15. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 15 IN THE BEGINNING… GRIDs Idea GRIDs Architecture GRIDs Business Model Computation/Data Information Knowledge

16. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 16 The GRIDs Architecture Knowledge Layer Information LayerComputation / Data Layer Data to Knowledge Control

17. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 17 The GRIDs Architecture Data to Knowledge Control Particle Physics ApplicationGenomics Application Environmental ApplicationE-Business Application

18. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 18 The GRIDs Architecture Computation / Data Layer Data to Knowledge Control

19. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 19 The Computation/Data Grid a computation / data grid –raw computing power –associated data stores –network-connected –both floating point computation and data-handling with logic;

20. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 20 The GRIDs Architecture Information Layer Data to Knowledge Control

21. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 21 The Information Grid an information grid superimposed –connecting together the major information sources –interfaces : homogeneous access to heterogeneous distributed information –sophisticated statistical analysis / reduction techniques for floating point numbers, textual information and multimedia information – special facilities for images –all with associated visualisation and VR (virtual reality) facilities.

22. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 22 The GRIDs Architecture Knowledge Layer Data to Knowledge Control

23. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 23 The Knowledge Grid a knowledge grid superimposed –utilising KDD (knowledge discovery in database) technology of which a well- known component is ‘data mining’. –support intelligent assists to decision makers (from control room to strategic thinkers) –provide interpretational semantics on the information.

24. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 24 In addition…... Each grid will have suitable security controls –information availability –prevention of unauthorised access appropriate to the source and the accessor. Similarly rights access (e.g. copyright, IPR) will be controlled.

25. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 25 IN THE BEGINNING… GRIDs Idea GRIDs Architecture GRIDs Business Model Computation/Data Information Knowledge

26. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 26 The GRIDs Business Model e-Science Pull IT Industry Supplies (with help from academia) Middleware and Metadata e-Business

27. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 27 The GRIDs Business Model Pull from science challenge projects Supply of common middleware and metadata –for the science –with industry - technology transfer use middleware and metadata for e-business push

28. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 28 CONTENTS What GRIDs Can Do In the Beginning A Possible Architecture Scenarios Conclusion

29. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 29 A Possible Architecture Proposed Architecture Requirement Metadata Representative Agents Brokers

30. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 30 PROPOSED ARCHITECTURE SOURCE RESOURCE Resource Agent User Agent User Metadata Source Metadata Source Agent The GRIDs Environment Resource Metadata USER

31. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 31 PROPOSED ARCHITECTURE Metadata Representative Agents Brokers User Source Resource User Source Resource

32. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 32 PROPOSED ARCHITECTURE Or more formally: Given U represents a User –with Um as metadata, Ua as agent and Uq as request Given S represents a source –with Sm as metadata and Sa as agent Given R represents a physical resource –with Rm as metadata and Ra as agent –(with a special R, labelled N for Network)

33. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 33 A Possible Architecture Proposed Architecture Requirement Metadata Representative Agents Brokers

34. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 34 Requirement U:USER S:SOURCE R:RESOURCE Rm:Resource Metadata Ra:Resource Agent Ua:User Agent Um:User Metadata Sm:Source Metadata Sa:Source Agent brokers Uq

35. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 35 Requirement The GRIDs problem, then, is to provide U with that subset of S required by Uq: (a) with appropriate precision and recall (b) subject to rights of access (including security, IPR, copyright, charging) (c) in an appropriate form (presentation in the W3C sense) (d) at an appropriate place (e) in a timely fashion

36. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 36 Requirement U:USER S:SOURCE R:RESOURCE Rm:Resource Metadata Ra:Resource Agent Ua:User Agent Um:User Metadata Sm:Source Metadata Sa:Source Agent brokers Uq

37. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 37 Requirement (a) precision/recall: intersection of all Sm and all S with Uq moderated by Um (b) rights: intersection of the selected Sm (representing selected S) with Um (c) form: intersection of Sm (presentation capability) with Um (presentation requirement) and may require specialist physical resources Rm (d) place: intersection of Um with Nm and may require specialist physical resources Rm to optimise based on the selected Sm (e) time: intersection of Um with Nm and may require specialist physical resources Rm to optimise based on the selected Sm

38. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 38 A Possible Architecture Proposed Architecture Requirement Metadata Representative Agents Brokers

39. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 39 Classification of Metadata data (document) SCHEMANAVIGATIONALASSOCIATIVE how to get it constrain it view to users

40. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 40 Metadata There is growing acceptance (especially in the W3C world) that Xm {X = any} can be divided into: –(1) schema metadata (constrains the data formally) example is a database schema or a DTD –(2) navigational metadata (provides access - may be a URL, could be a database view or query) –(3) associative metadata (related to the resource being described and divided into: (i) descriptive metadata: e.g. library catalogue card (ii) restrictive metadata: eg rights access (iii) supportive metadata: eg dictionaries, thesauri, domain ontologies)

41. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 41 User Metadata Schema metadata: Signature / identifying information for authentication Navigational Metadata: access - eg IP address, mobile phone number Associative Metadata –descriptive: Preferences for eg presentation, payment, credentials for rights –restrictive: personal rights / claims eg privacy –supportive: eg organisation taxonomy

42. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 42 Source Metadata Schema metadata: quality Navigational Metadata : access eg URL Associative metadata –descriptive : assisting query –restrictive : rights, charges –supportive : thesauri, ontologies assisting query and interpretation / explanation of results

43. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 43 Resource Metadata Schema Metadata: quality Navigational Metadata access eg IP address Associative metadata –descriptive: resource capability –restrictive: rights of access, charges –supportive: hints for use

44. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 44 A Possible Architecture Proposed Architecture Requirement Metadata Representative Agents Brokers

45. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 45 Representative Agents Represent the entities {U, S, R} continuously and actively within the GRIDs environment With metadata represent the entity to others represented by their agents Act on behalf of the entity

46. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 46 The Agents Working U:USER S:SOURCE R:RESOURCE Rm:Resource Metadata Ra:Resource Agent Ua:User Agent Um:User Metadata Sm:Source Metadata Sa:Source Agent brokers Uq

47. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 47 The Agents Working Ua (the user agent) –accepts Uq (the request) –and, using Um (user metadata) –negotiates through brokers with Sa and Ra (agents for source(s) and physical resource(s)) –themselves using Sm and Rm (metadata) to satisfy Uq

48. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 48 A Possible Architecture Proposed Architecture Requirement Metadata Representative Agents Brokers

49. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 49 Brokers (a) authentication, (b) clarification / precision of request, (c) resource discovery (information and if necessary compute power, visualisation facilities etc) (d) authorisation (rights), (e) offer and pricing, (f) closure of deal (U accepts (e))

50. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 50 Brokers (continued) (g) fusion of responses, (h) application of any transformation / analysis / simulation / visualisation processes, (i) presentation formatting (for variously abled devices and people using various resources), (j) network routing, and (k) scheduling of physical resource access / usage

51. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 51 Brokers U:USER S:SOURCE R:RESOURCE Rm:Resource Metadata Ra:Resource Agent Ua:User Agent Um:User Metadata Sm:Source Metadata Sa:Source Agent brokers Uq

52. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 52 Monitoring Brokers and others will monitor –quality of service, –utilisation of resource collections –specialist physical resources –etc etc.

53. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 53 CONTENTS What GRIDs Can Do In the Beginning A Possible Architecture Scenarios Conclusion

54. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 54 SCENARIOS U interacts (Uq) with Ua (with its Um) Ua uses brokers and Sa (with its Sm) and Ra (with its Rm) to clarify request Ua negotiates deal; User confirms acceptance and request executed Results - after any fusion, analysis and visualisation - to Ua for presentation (mode defined by Um) to U

55. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 55 Scenarios: Initiation (U: Uq) U:USER S:SOURCE R:RESOURCE Ua:User Agent Um:User Metadata Uq

56. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 56 SCENARIOS U interacts with Ua (with its Um) Ua uses brokers and Sa (with its Sm) and Ra (with its Rm) to clarify request Ua negotiates deal; User confirms acceptance and request executed Results - after any fusion, analysis and visualisation - to Ua for presentation (mode defined by Um) to U

57. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 57 Scenarios: UA uses Brokers U:USER S:SOURCE R:RESOURCE Rm:Resource Metadata Ra:Resource Agent Ua:User Agent Sm:Source Metadata Sa:Source Agent Clarify Uq

58. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 58 SCENARIOS U interacts with Ua (with its Um) Ua uses brokers and Sa (with its Sm) and Ra (with its Rm) to clarify request Ua negotiates deal; User confirms acceptance and request executed Results - after any fusion, analysis and visualisation - to Ua for presentation (mode defined by Um) to U

59. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 59 Scenarios: Ua makes deal U:USER S:SOURCE R:RESOURCE Rm:Resource Metadata Ra:Resource Agent Ua:User Agent Um:User Metadata Sm:Source Metadata Sa:Source Agent deal accept

60. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 60 SCENARIOS U interacts with Ua (with its Um) Ua uses brokers and Sa (with its Sm) and Ra (with its Rm) to clarify request Ua negotiates deal; User confirms acceptance and request executed Results - after any fusion, analysis and visualisation - to Ua for presentation (mode defined by Um) to U : Ua pays

61. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 61 Scenarios: Results U:USER S:SOURCE R:RESOURCE Rm:Resource Metadata Ra:Resource Agent Ua:User Agent Um:User Metadata Sm:Source Metadata Sa:Source Agent Fusion / analysis Result /mode

62. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 62 OTHER SCENARIOS Ua acts as answering machine, redirects messages as email, sms, voice.. Ua acts as diary manager negotiating meetings, dates, times, travel, accommodation… Sa advertises content (using Sm) Ra advertises resource capability (using Rm)

63. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 63 CONTENTS What GRIDs Can Do In the Beginning A Possible Architecture Scenarios Conclusion

64. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 64 Beyond the World Wide Web WWW and GRIDs limited computation limited information access style not meaning limited security browsing laborious for user computation heterogeneous distributed database meaning and style authentication / authorisation purposeful transactions works for user

65. ©CLRC/ITD/Keith G JefferyEuroCRIS May 2001: Introduction to GRIDs May 2001 65 e-Everything and GRIDs e-Science Pull IT Industry Supplies (with help from academia) Middleware and Metadata e-Business e-Environment e-Culture e-Learning e-Health