1. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu NaradaBrokering: A Middleware Framework and Architecture for Enabling P2P Grids Middleware 2003 Rio de Janeiro, Brazil 16-20 June 2003 Shrideep Pallickara and Geoffrey Fox spallick, gcf@indiana.edu Community Grid Computing Laboratory, Pervasive Technology Labs Indiana University. http://www.naradabrokering.org

2. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu Talk Outline P2P Grids Messaging Infrastructure requirements NaradaBrokering Overview P2P support Extensible transport framework Security and Monitoring Infrastructure Conclusions Future Work

3. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu P2P Grids Grids - Typified by infrastructure for seamless access to high-end computing resources. –Job submissions, data management services. P2P systems – Sophisticated resource sharing environments. –Search, discovery & sharing of resources (CPU/files). P2P Grid comprises services of both Grids and P2P systems. Integrates ideas of computational grids, web services, P2P systems and message-oriented-middleware.

4. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu Messaging Infrastructure for P2P Grids Scaling – Support large number of devices/users. Efficient disseminations of interactions Guaranteed Delivery Mechanisms Location Independence Support for P2P interactions – JXTA, Gnutella Interoperate with Messaging clients Performance Monitoring and Security services.

5. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu NaradaBrokering: Features Based on a network of cooperating broker nodes –Cluster based architecture allows system to scale to arbitrary size Originally to provide uniform software multicast to support real-time collaboration linked to publish- subscribe for asynchronous systems. Now has four major core functions –Message transport (based on performance) in multi-link fashion –General publish-subscribe including JMS, JXTA and Gnutella (started) –Support for RTP-based audio/video conferencing. –Federation of multiple instances (just starting) of Grid services

6. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu NB: Engineering Issues Addressed Tunnel through firewalls/proxies –Microsoft’s ISA, Checkpoint, Apache Support for multiple network protocols such as TCP, UDP, Multicast, SSL, RTP and HTTP. Support for both blocking and non-blocking IO. Scaling of software multicast –Efficient calculation of destinations and routes.. Supports local broker accesses Transparently replace single server JMS systems with a distributed solution.

7. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu NaradaBrokering: Organization

8. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu Organization of Profiles and Routing Client subscriptions are stored hierarchically within the system. –A broker maintains client subscriptions, cluster-controller maintains broker subscriptions and so on. When an event is received, the event is matched against stored profiles and destinations are computed –A cluster-controller computes broker destinations. A broker computes client destinations. Every broker node, when supplied with a set of destinations, computes the best broker-hops to take to reach these destinations.

9. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu NaradaBrokering: Distributed Results Every broker – SPARC Ultra-5 (128 MB, 333 MHz) 105 Clients – SPARC-Ultra-60 (512MB, 360 MHz) JRE-1.2.1, 100 Mbps Network

10. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu NaradaBrokering: Matching Engines Matching engines are responsible for matching events against stored profiles. NaradaBrokering supports a variety of matching engines supporting –“/” separated String based topics –Equality based topics –Integer based topic –SQL (from JMS) –XPath based queries

11. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu Matching Engine Performance Stand alone process Pentium-3 1 GHZ 256MB RAM JRE 1.4

12. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu Why Support P2P? Core features – Resource Sharing & Discovery CPU cycles: SETI@home, Folding@HOME File Sharing: Napster, Gnutella Deployments user driven – No dedicated management Management of resources –Expose resources & specify security strategy –Replicate resources based on demand Dynamic peer groups, fluid group memberships Sophisticated search mechanisms –Peers respond to queries based on their interpretations –Responses do not conform to traditional templates.

13. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu P2P Systems: The Downsides Routing not very sophisticated –Inefficient network utilization Usually relying on simple forwarding of interactions –Peer Traces (to eliminate echoing) –Attenuations (to suppress propagation) TTL’s associated with interactions. Interactions are attenuated –Resulting in localized interactions and a fragmented world of multiple P2P subsystems

14. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu NaradaBrokering & JXTA Interaction Model Based on proxy model –Acts as both Rendezvous peer (JXTA routers) and NaradaBrokering client. No changes to JXTA core or straitjacketing of interactions –Change made to Rendezvous layer Peers are not aware that they interact with a Narada-JXTA proxy or Rendezvous peer. Narada-JXTA provides JXTA guaranteed long distance delivery

15. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu NaradaBrokering-JXTA Proxy Glean relevant information from JXTA interactions. –Peer group advertisements (XML Doc describing resource) –Requests/Responses to be part of peer group. –Messages sent to a peer group. –Queries and responses to these queries. Subscribe to relevant topics to ensure delivery Construct corresponding Narada-JXTA event from interactions. –These events lend themselves to efficient routing.

16. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu NaradaBrokering-JXTA Apps and Setups Applications –Integrated NaradaBrokering-JXTA environment tested under JXTA shell and myJxta (InstantP2P) –Plan to integrate myJxta into Anabas (distance education software) with NaradaBrokering managing P2P and middle- tier (JMS) style interactions. Experimental Setup –Sender/receiver - (Pentium-3, 1 GHz, 256 MB RAM). –Every node (broker/router) hosted on a different machine (Pentium-3, 1 GHz, 256 MB RAM). –Machines reside on a 100 Mbps LAN –Run-time environment for all the processes is JDK-1.3 build Blackdown-1.3.1, Red Hat Linux 7.3

17. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu

18. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu NaradaBrokering Communications Applications interface to NaradaBrokering through UserChannels. UserChannels have publish/subscribe semantics Links implement a single conventional “data” protocol. –Different links can have different underlying transport implementations –Addition of new transport protocols within the Framework is easy to achieve. –Administrative channel negotiates the best available communication protocol Link implementations can incorporate their own handshaking protocols to facilitate communications and data exchange.

19. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu NaradaBrokering Communications - II

20. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu Performance Monitoring Every broker incorporates a Monitoring service that monitors links originating from the node. Every link measures and exposes a set of metrics –Average delays, jitters, loss rates, throughput. Individual links can disable measurements for individual or the entire set of metrics. Measurement intervals can also be varied Monitoring Service, returns measured metrics to Performance Aggregator.

21. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu Performance Aggregation Aggregated information will be used to –Circumvent bottlenecks –Aid routing algorithms –Facilitate Dynamic Load-balancing

22. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu

23. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu NaradaBrokering: Security Framework Based on Message Level Security Authentication – Confirm whether a user is really who he says he is. Authorization – Identify if the user is authorized to receive certain events Key distribution – Based on authentication & authorization distribute keys, which ensure that only the valid clients are able to decrypt encrypted data. Digital Signing – Have the ability to verify the source of the event and whether the source is authorized to publish events conforming to the specified template. Communication Protocol Independent Detection and Response to Security Compromise –Clients required to respond to queries (stored during initializations) issued at random intervals.

24. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu

25. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu Security Results Experiments performed on a Pentium-3, 1.5GHz, 512 MB RAM. JRE 1.4.1, Cryptographic provider is IAIK Points in the graphs represent the average value of the operation being performed 1000 times. Results indicate that the security scheme does not introduce unacceptable delays. –Since messages are encrypted only once, costs are amortized during traversal over multiple broker hops.

26. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu

27. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu

28. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu Conclusions NaradaBrokering is a messaging infrastructure that is appropriate for P2P Grids. Results demonstrate that it can be used for synchronous and asynchronous applications. Availability of multiple matching engines provides for sophisticated interactions between entities within the system.

29. June 18 th 2003. ACM Middleware 2003 http://www.naradabrokering.org spallick,gcf@indiana.edu Future Work Federation of P2P systems and accompanying search/query/response mechanisms. –Ongoing work with Limewire (Gnutella) Distributed A/V conferencing management. Dynamic Resource Management Management of lightweight XML databases –Ongoing investigations with Apache Xindice and Source Forge eXist.