1. ShareGrid: architettura e middleware
Cosimo Anglano, Massimo Canonico, Marco Guazzone
Distributed Computing Systems Research Laboratory
Dipartimento di Informatica
Universita’ del Piemonte Orientale
Alessandria

2. Incontro su ShareGrid, 5/12/2007, Torino
Outline
Grid Computing panorama
ShareGrid architecture
The OurGrid middleware
ShareGrid applications
Conclusions and future works
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

3. Incontro su ShareGrid, 5/12/2007, Torino
Grid computing
E-Science: massive use of computers to perform scientific research
investigation tools (simulation, data mining, etc.)‏
remote collaboration tools
Grid computing conceived as the answer to these computation needs:
use of a set of geographically dispersed resources as a single computing platform
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

4. Incontro su ShareGrid, 5/12/2007, Torino
Classical Grids
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

5. Incontro su ShareGrid, 5/12/2007, Torino
Classical Grids
Works for you if:
you know other people/institutions with similar needs
you have “enough” resources to contribute
you have enough skilled sysadms to configure and maintain your resources
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

6. Volountary-Computing Grids
e.g.
BOINC most prominent middleware
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

7. Voluntary Grid Computing
Works only if you
have a very good support team to run “the server”
invest a good deal of effort in “advertising”
have a very high visibility project
are in a prestigious institution
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

8. Incontro su ShareGrid, 5/12/2007, Torino
Is a peer-to-peer grid
participants join spontaneously, without prior agreements/negotiations, and may leave without prior notice
focuses on cooperative resource sharing: “I will let you use my resources (when I don’t need them) if I can use yours”
based on the OurGrid middleware
carried on at the Universidad Federal de Campina Grande (
sponsored by HP Brazil
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

9. ShareGrid (OurGrid) architecture
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

10. ShareGrid (OurGrid) architecture
OurGrid P2P network
Peer: Site Manager
Grid-wide Resource Sharing
Resource Broker for submitting users
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

11. ShareGrid (OurGrid) architecture
MyGrid: User Interface &
Application Scheduling
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

12. ShareGrid (OurGrid) architecture
Worker node (gum agent)
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

13. ShareGrid (OurGrid) architecture
SWAN: Sandboxing
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

14. Incontro su ShareGrid, 5/12/2007, Torino
OurGrid Properties
Easy to join
just download the peer and the worker agent code
Easy to install, configure and program
No need for specialized support team
Easy to use
MyGrid is simple to install and use
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

15. Incontro su ShareGrid, 5/12/2007, Torino
OurGrid Properties
Clear incentive to join the system
One can’t be worse off by joining the system
Noticeable increased response time
Free-riding resistant
Basic dependability properties
Some level of security (SWAN)
Automatic resubmission of failed tasks
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

16. ShareGrid (OurGrid) application model
Restricted to Bag-of-Tasks (BoT) applications
no communication among tasks
Many applications fall in the BoT class:
data mining
massive search
bio computing
parameter sweep
Monte Carlo simulations
fractal calculations
image processing
...
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

17. OurGrid: security mechanisms
SWAN: Xen virtual machine, with no network access, and disk access only to a designated partition
leverage the fact that BoT applications only communicate to receive input and return the output
input/output is done by OurGrid itself that runs in a Xen virtual machine
SWAN is optional:
a few minutes to required to boot and shutdown the virtual machine
requires specific hw/sw configuration (Linux and hw support for virtualization)
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

18. OurGrid: fostering cooperation
To avoid selfish behaviours (a.k.a. free riding) OurGrid uses the Network of Favours mechanism
All peers maintain a local balance for all known peers
Peers with greater balances have priority
Newcomers and peers with negative balance are treated equally
The emergent behaviour of the system is that by donating more, one gets more resources back
No additional infrastructure is needed
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

19. ShareGrid: issues and solutions
Software dependencies
Applications require the availability of specific library
Current solution:
installation of dynamic version of the libraries OR
static compilation of applications (requires source code)
Next solution:
Encapsulation of all execution environment required by each application into a virtual machine
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

20. ShareGrid: issues and solutions
Input and output data
all the data required and produced by an application must be staged in and out for each task
current solution:
OurGrid transparently copies the data to and from the worker node
Storage affinity scheduling policy
Next solution:
Integration of a file transfer system
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

21. ShareGrid: issues and solutions
Data privacy
some application process confidential or sensitive data
Current solution:
none
Next solution:
Use of encrypted file system to store data
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

22. ShareGrid: issues and solutions
The system does not manage itself
One of the most annoying bug consists in unpredictable disappearance of a peer
Current solution:
Restarting the peer
Next solution:
Software monitor able to check the availability of each peer and automatically restart those are down
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

23. ShareGrid: issues and solutions
Outbound network access
peers must be able to go through a firewall (if present)
sometimes a laboratory may not be allowed to directly configure the corresponding firewall
solution: if connected to the same network of another entity that has outbound access, create a Virtual Private Network
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

24. ShareGrid: issues and solutions
The world is not Linux-centric
OurGrid has a strong Linux orientation: peer and client processes run only on *nix machines
Next solution
none for the peer
web portal providing access to ShareGrid (under development)
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

25. Incontro su ShareGrid, 5/12/2007, Torino
Conclusion
ShareGrid is a peer-to-peer desktop grid aiming at aggregating and sharing, in a very simple and transparent way, computing resources
Good usability
Rapid turnaround time
Good satisfaction of the users
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino

26. Incontro su ShareGrid, 5/12/2007, Torino
Future work
Integration with general-purpose virtualization (go beyond SWAN)
Integration of a better storage support
Better support for data security and privacy
Cosimo Anglano
Incontro su ShareGrid, 5/12/2007, Torino