1. XtremWeb : resolving deployment Oleg Lodygensky, Miron Livny, Franck Cappello, Vincent Neri LRI - Université Paris sud

2. XtremWeb : resolving deployment
Summary
XtremWeb
Condor deployment
Jxta deployment
Applications
14 novembre novembre 2018
XtremWeb : resolving deployment

3. XtremWeb : resolving deployment
In a nutshell
peer 2 peer distributed computing
Strong Security (Sandbox, SSL, Coordinator authentication)
platform independent (Linux, Windows, MacOS)
application checkpointing
binary migration
cycle stealing mode
Pull mode for task dispatching
Each peer can be an XW dispatcher, client or worker.
All peers wait for a coordinator.
14 novembre novembre 2018
XtremWeb : resolving deployment

4. XtremWeb 1.0.5 XtremWeb entities coordinator check point server
monitor
workers
clients
Worker
Internet
Coordinator
XtremWeb network (XWN)
CheckPoint Server
Client
Each peer can be an XW dispatcher, client or worker.
All peers wait for a coordinator.
It’s now a
full production
platform
14 novembre novembre 2018
XtremWeb : resolving deployment

5. XtremWeb Network
It’s a network in the sens that everything is structured toward a single goal.
Tasks management policy :
restart lost tasks (eventually from last check point)
untrusted tasks (sandbox)
untrusted hosts (SSL)
Worker
Internet
Coordinator
XtremWeb network (XWN)
CheckPoint Server
Client
Each peer can be an XW dispatcher, client or worker.
All peers wait for a coordinator.
Hosts dispatched over Internet
14 novembre novembre 2018
XtremWeb : resolving deployment

6. XtremWeb protocols Pull mode only : workers initiate each dialog.
Upgrade
Worker
Coordinator
workResult
host Register
Work
WorkRequest
Params
Alive
Each peer can be an XW dispatcher, client or worker.
All peers wait for a coordinator.
Even upgrade: If this is decided by server, it is still at the initiative of the worker.
14 novembre novembre 2018
XtremWeb : resolving deployment

7. XtremWeb upgrade protocol
Upgrade is a special case : the worker forks a new process with new provided code and kill itself.
Worker
(PID= X)
Download
Upgrade
(PID= Y)
Fork
Stop
Each peer can be an XW dispatcher, client or worker.
All peers wait for a coordinator.
Process management should take care that PID changes!
14 novembre novembre 2018
XtremWeb : resolving deployment

8. XtremWeb 1.0.6 networks communication
XWN
internet
XWN’
Server may release worker
Server may ask for more workers to each others
Workers belongs to one server at a time
Workers leave server on specific order only
Workers still don’t know each others
Action spécifique
PPF
Make XtremWeb networks interoperables
14 novembre novembre 2018
XtremWeb : resolving deployment

9. Paris-South Paris-South Paris-South Wisc/Mad Firewall Chalenge
Encrypted coms. XW
Coord
Paris-South XW
Coord
Action spécifique
PPF XW
Worker XW
Worker XW
Worker XW
Worker XW
Worker XW
Worker
Condor Pool
Condor Pool
Paris-South
Wisc/Mad
14 novembre novembre 2018

10. XtremWeb : resolving deployment
Summary
XtremWeb
Condor deployment
Jxta deployment
Applications
14 novembre novembre 2018
XtremWeb : resolving deployment

11. XtremWeb : resolving deployment
Condor
University of Wisconsin-Madison
In a nutshell
specialized batch system for compute-intensive jobs
Matchmaking, master/worker, DAGman…
platform independent
application checkpointing
binary migration
cycle stealing mode
Push mode for task dispatching
Each peer can be an XW dispatcher, client or worker.
All peers wait for a coordinator.
14 novembre novembre 2018
XtremWeb : resolving deployment

12. Condor pool Tasks management policy:
restart lost tasks (eventually from last check point)
trusted tasks (security is under host administrator responsability)
trusted hosts
(UID/GID and/or PIK)
Each peer can be an XW dispatcher, client or worker.
All peers wait for a coordinator.
Master
Worker
Intranet
Condor Pool
(CPL)
CheckPoint Server
All hosts included in a single
administration domain
14 novembre novembre 2018
XtremWeb : resolving deployment

13. Condor onto Grid Using Globus (a Grid Toolkit) to:
connect several Condor pools (several admin domains)
global administration relies on Globus (GRAM, GASS and GSI)
Internet
Private Net
Client
Globus
CPL
Each peer can be an XW dispatcher, client or worker.
All peers wait for a coordinator.
Pools dispatched over multiple
administration domains
But using Globus increases system complexity & deployment
14 novembre novembre 2018
XtremWeb : resolving deployment

14. XtremWeb : resolving deployment
Condor vs XtremWeb
Condor
batch system
security delegated
over intranet
matchmaking (resource management)
meta tasks
several pools installed
XtremWeb
P2P dist. computing
security included
over internet
Each peer can be an XW dispatcher, client or worker.
All peers wait for a coordinator.
14 novembre novembre 2018
XtremWeb : resolving deployment

15. XtremWeb-C (XW in Condor) Deploying XW Workers with Condor
Dispatch XtremWeb workers as Condor tasks
Condor pool gets :
pull mode task dispatching
sandboxing + SSL
Master
Intranet
Condor Pool for XtremWeb
(XWP)
Worker
Each peer can be an XW dispatcher, client or worker.
All peers wait for a coordinator.
XtremWeb network gets :
deployment
pool management
14 novembre novembre 2018
XtremWeb : resolving deployment

16. Condor-XW: Connecting & Coordinating Condor Pools with XW
Instanciate an XtremWeb server and workers.
Condor pool gets :
accept hosts not included in any pool
a lightweight Grid
Internet
XWP
Worker
Coordinator
Each peer can be an XW dispatcher, client or worker.
All peers wait for a coordinator.
XtremWeb network gets :
decentralized pool of hosts
14 novembre novembre 2018
XtremWeb : resolving deployment

17. Condor submit description file#
# File : worker.cmd
# Purpose : Condor commande file for XtremWeb workers
universe=vanilla
Executable=worker.sh
output=xwwworker-$(Process).out
error=xwwworker-$(Process).err
log=xwwworker-$(Process).log
queue
Using vanilla since
java universe bugged in
Condor 6.4.0!!!
6.4.3 just released;
under LRI
Each peer can be an XW dispatcher, client or worker.
All peers wait for a coordinator.
14 novembre novembre 2018
XtremWeb : resolving deployment

18. Workers script for upgrade
#!/bin/sh
# File : worker.sh
# Purpose : this script is intended to be used with condor
# it checks error code; 3 means restarts
# This is for Condor vanilla universe only!
restart=1
while [ $restart -eq 1 ]; do
java -jar XWWLaunch.jar -f xwrc
if [ $? -ne 3 ]; then
restart=0
else
echo Restarting... fi
done
Each peer can be an XW dispatcher, client or worker.
All peers wait for a coordinator.
14 novembre novembre 2018
XtremWeb : resolving deployment

19. Condor submit description file#
# File : worker.cmd
# Purpose : Condor commande file for XtremWeb workers
universe=java
Executable=XWWLaunch.jar
Parameters=Launcher
output=xwwworker-$(Process).out
error=xwwworker-$(Process).err
log=xwwworker-$(Process).log
queue
Each peer can be an XW dispatcher, client or worker.
All peers wait for a coordinator.
14 novembre novembre 2018
XtremWeb : resolving deployment

20. Deployment (kind of testbed…)
Madison/
Winsconsin
Condor Pool
First phase:
use LRI server only
deploy workers
use Madison pool (400 hosts)
log each
draw conclusions
u-psud
network
math.u-psud.fr
mathens.u-psud.fr
lri.fr
others labs
lal.in2p3.fr
in2p3
ipno.in2p3.fr
internet
Action spécifique
PPF
14 novembre novembre 2018
XtremWeb : resolving deployment

21. XtremWeb : resolving deployment
Next step : (as soon as previous concluded)
deploy servers + workers
log each
XWN
internet
XWN’
Some
Condor Pool
Action spécifique
PPF
14 novembre novembre 2018
XtremWeb : resolving deployment

22. XtremWeb : resolving deployment
Summary
XtremWeb
Condor deployment
Jxta deployment
Applications
14 novembre novembre 2018
XtremWeb : resolving deployment

23. Jxta Jxta is a proposal for a framework that defines
Sun Inc.
Jxta is a proposal for a framework
that defines
a set of protocols
to standardize development
of Open Network Platforms
for P2P computing.
14 novembre novembre 2018
XtremWeb : resolving deployment

24. Jxta : Benefits Jxta proposes : decentralization (Jxta services)
dynamicity (Jxta advertisements)
connectivity (Jxta peer discovery)
scalability (Jxta groups)
Jxta protocols could help to make
one step toward Grid!
We must stay in the race: there’s allready
a proposal for global computing with Jxta [3]
But for Java classes migration only! 
14 novembre novembre 2018
XtremWeb : resolving deployment

25. Jxta : topology Endpoints (or routers) help to bypass Firewalls
World peer group
Peer group A
Peer
Some peers have special roles
Peer
Rendez vous Peer
Peer group B
Peer
Endpoint Peer
All peer is member of at least one group : « World group »
Endpoints (or routers) help to bypass Firewalls
Rendez vous propagates known advertisements
Member of
three groups
14 novembre novembre 2018
XtremWeb : resolving deployment

26. Jxta : tech. overview XML
Groups publish advertisements.
Everything is based on advertisements; this enables visibility.
Group Ad
Peers publish advertisements.
Group Service Ad
Service Ad
Peer Ad
It does not mean
unsecured aperture :
security has to be
implemented at peer
or group level.
Peer
Peer
Other Ad
Other Ad
Peers communicate
a group
World peer group
Advertisements help deployement and dynamicity.
14 novembre novembre 2018
XtremWeb : resolving deployment

27. Jxta : Comparaisons Jini Jxta Jxta .NET Soap Jxta Web services Jxta
Centralized
RMI + serialization
Java
full P2P
XML
Java, C, C++, python…
full P2P architecture
Mainly client/serveur architecture
Neither P2P tranport nor discovery
Soap
Jxta
Web services
Jxta
Remote service Access
XML
Multi languages
RSA + others (pipes…)
XML
Multi languages
centralized (UDDI)
HTTP/XML
TCP-IP
full P2P
XML
transport agnostic
Community is building Soap on top of JXTA
14 novembre novembre 2018
XtremWeb : resolving deployment

28. XtremWeb : resolving deployment
Jxta : Comparaisons
Jxta
Web services
14 novembre novembre 2018
XtremWeb : resolving deployment

29. Universal Binding Mechanism
All traditional “binding” operations in the JXTA network are implemented as a search for an advertisement.
Examples:
DNS (searching for Peer or PeerGroup advertisements)
Service Location (searching for Module advertisements)
Directory Service (searching for Peer advertisements)
Socket Binding (searching for Pipe advertisements)
File systems (searching for Content advertisements)
=> No Centralization Required !!
JXTA does not exactly mandate how discovery is done. It may centralized, decentralized or even a hybrid of both.
14 novembre novembre 2018
XtremWeb : resolving deployment

30. XtremWeb + Jxta Without Jxta With Jxta
XWN
internet
XWN’
World peer group
XW group
XW group
Worker
Coordinator
Worker
Worker
Client
Worker
Coordinator
Client
There might be several XW groups.
A peer can be member of different groups with a different role on each.
Worker
Dynamic topology :
non static roles
roles given on the fly
Use Jxta for discovery, connectivity and topology
14 novembre novembre 2018
XtremWeb : resolving deployment

31. XtremWeb : resolving deployment
Summary
XtremWeb
Condor deployment
Jxta deployment
Applications
14 novembre novembre 2018
XtremWeb : resolving deployment

32. Exploring high energy cosmic rays
LABORATOIRE de l’ACCELERATEUR LINEAIRE
Detect and study 1019 eV cosmic rays
The world’s largest cosmic ray detector
$100M (30% USA; 30% South America; 40% others)
More than 40 institutes in the collaboration
Computing
100Mb/Day expected
hours simulation time
14 novembre novembre 2018
XtremWeb : resolving deployment

33. Exploration of conformational transitions in proteins
INSTITUT de BIOCHIMIE et de BIOPHYSIQUE MOLECULAIRE et CELLULAIRE
Comprendre/maitriser les paramètres de stabilité et d’expression d’activité d’une protéine.
Analyse des propriétés dynamiques de la structure protéique.
Evaluation des facteurs de stabilité et de perturbation du processus de repliement par mutations.
Approche expérimentale : mutagénèse de protéines modèles.
Modélisation moléculaire: explorer les possiblités conformationnelles des macromolécules à des échelles de temps > 1nanosec et les appliquer aux protéines étudiées au laboratoire.
14 novembre novembre 2018
XtremWeb : resolving deployment

34. Exploration of conformational transitions in proteins
"interesting"
conformation
(amyloid?)
e.g., normal
prion protein
Molecular Dynamics is great for simulating random thermal deformations of a protein...
but unlikely to reach a particular conformation of the protein, even if you really want to
Vibrational Modes is great for identifying preferred deformations towards "interesting" conformations
but strictly applicable to small deformations only
Combined approach: we force molecular dynamics to explore "interesting" deformations identified by vibrational modes
David Perahia and Charles Robert
UMR8619 CNRS
University of Paris-Sud Orsay France
14 novembre novembre 2018
XtremWeb : resolving deployment

35. Obtain free-energy profiles
Explore low-energy (favorable) transition pathways
Extend to multiple dimensions (energy surfaces)
energy barrier
4) Calculate free energy profile
...
etc ...
3) Gather statistics
XtremWeb
Workers +
Condor
Pools
XtremWeb + Condor
2) Perform m constrained molecular dynamics simulations for each (= nm workers )
David Perahia and Charles Robert
UMR8619 CNRS
University of Paris-Sud Orsay France
1) Generate n starting conformations along coordinate of interest
14 novembre novembre 2018
deformation coordinate
XtremWeb : resolving deployment

36. XtremWeb : resolving deployment
References
Condor :
XtremWeb :
Auger :
IBBMC :
Jxta :
[1] « The Grid : Blueprint for a new Computing Infrastructure », I. Foster et C. Kesselman, Morgan-Kaufmann
[2] « Java P2P unleashed », R.Flenner, M.Abbott, F.Cohen, N.Krishnan, A.Moffet, R.Ramamurti, B.Siddiqui, F.Sommers
[3] « Framework for Peer-to-Peer Distributed Computing in a Heterogeneous, Decentralized Environment » Jerome Verbeke, Neelakanth Nadgir, Greg Ruetsch, Ilya Sharapov Sun Microsystems, Inc., Palo Alto, CA 94303
[4] « Web services and Jxta : companion technologies »
by Bradley S. O'Hearne
14 novembre novembre 2018
XtremWeb : resolving deployment