25/05/98Stéphane Boyera, Luc Ottavj1 W3C-LA Web Mirroring Activity

25/05/98Stéphane Boyera, Luc Ottavj2 Plan n Web Mirroring Keywords n Underlying Problems of Mirroring n Experimental Protocol n Hardware n Software n Results n Conclusion

25/05/98Stéphane Boyera, Luc Ottavj3 Web Mirroring Keywords n Transparency for the user n Homogeneity between mirrors n Independence n Fault Tolerance and flexibility

25/05/98Stéphane Boyera, Luc Ottavj4 Underlying Problems of Mirroring n Building the package to send n Defining the periodicity of update n Broadcasting the update n Recovering local or global failures

25/05/98Stéphane Boyera, Luc Ottavj5 Periodicity of updates n True Real Time : –Initiative of mirroring given to the user n Simulated Real Time : –Initiative to the system –Short periodicity : ~30mn - 1 hour n Differed mirroring : –Initiative to the system –Periodicity : ~daily

25/05/98Stéphane Boyera, Luc Ottavj6 How is an update processed ? n Classical mirroring systems : rdist, ftp, … –A master server contact each mirror one by one –Drawbacks : the more numerous the mirrors the more decreased the perf. of the overall system n Reliable Multicast technology –Advantage : not affected by the number of mirrors –Drawbacks : no standard available but products

25/05/98Stéphane Boyera, Luc Ottavj7 The Underlying Network n Internet : actually not reliable n Satellite transmission : –Dedicated Bandwidth –Private Network –Well fitted for the multicast technology –Covering the whole continent

25/05/98Stéphane Boyera, Luc Ottavj8 Experimental Protocol n Building of update : endless rough process finding files to broadcast (~20mn) n Periodicity : Simulated Real Time n Broadcast : –using multicast through Internet –using multicast through Satellite –using unicast/TCP through Internet

25/05/98Stéphane Boyera, Luc Ottavj9 Hardware n On the Master Site (Sophia) –One master server housing files and being both multicast sources and unicast server –One Internet Multicast Router –One Satellite Router (Uplink) n On Mirror Sites –One Multicast router for both Internet and Satellite multicast groups –One server receiving the update by the three ways

25/05/98Stéphane Boyera, Luc Ottavj10 Software n Unicast / TCP : RSYNC –Free software –Implementing a gdiff algorithm n Multicast : Starburst Communication Mftp product

25/05/98Stéphane Boyera, Luc Ottavj11 Results n Period : 02/26/1998 to 05/11/1998 n Effective Working Days : 53 n Measures : 3653 n Transfers : 1218 (0,95 Transfer/hour) n Files : n Bytes : 0.54 GBytes (W3C site : 1.6 GBytes) n Average Update Size : 442 KBytes

25/05/98Stéphane Boyera, Luc Ottavj12 Rsync Results n Time used : s n Number of failures : 97 / 4872 n Average time to transmit : 231 s n Average Bandwidth : 15.3 Kbits/s

25/05/98Stéphane Boyera, Luc Ottavj13 Mftp Internet Results n Time Used : s n Number of Failures : 692 / 4872 n Average Time to transmit : 33 s n Average Bandwidth : 109 Kbits/s

25/05/98Stéphane Boyera, Luc Ottavj14 Mftp Satellite Results n Time Used : s n Number of Failures : 469 / 4872 n Average Time to transmit : 18 s n Average Bandwidth : 196 Kbits/s

25/05/98Stéphane Boyera, Luc Ottavj15 Conclusion n Mftp through Internet : Not Usable n Rsync : Surprisingly good –The most reliable solution –Can fit with the actual need (servers and files change rate) –Not performant –Weak supervision needed –Few Hardware

25/05/98Stéphane Boyera, Luc Ottavj16 Conclusion (2) n Mftp + Satellite : not fully operational –High performance –Good Scalability : n if number of servers increases n if traffic of the W3C site increases n if internet connectivity decreased –Accurate supervision needed –Low Reliability : n Operational Vs Experimental Uplink n Hardware problems with Comatlas card –Price ??