Presentation is loading. Please wait.

Presentation is loading. Please wait.

Of Rostock University DuDE: A D istributed Computing System u sing a D ecentralized P2P E nvironment The 4th International Workshop on Architectures, Services.

Published byJade Stephens Modified over 8 years ago

Similar presentations

Presentation on theme: "Of Rostock University DuDE: A D istributed Computing System u sing a D ecentralized P2P E nvironment The 4th International Workshop on Architectures, Services."— Presentation transcript:

1 of Rostock University DuDE: A D istributed Computing System u sing a D ecentralized P2P E nvironment The 4th International Workshop on Architectures, Services and Applications for the Next Generation Internet (WASA-NGI-IV) Bonn, Germany, October 4th, 2011 J. Skodzik, P. Danielis, V. Altmann, J. Rohrbeck, D. Timmermann University of Rostock, Germany Institute of Applied Microelectronics and Computer Engineering T. Bahls, D. Duchow Nokia Siemens Networks Broadband Access Division Greifswald, Germany

2 Outline Introduction & Motivation DuDE in General The DuDE Algorithm in Detail Test Scenario and Evaluation Summary and Future Work 2

3 Increasing number of Internet users and traffic data Internet Service Providers (ISPs) want to ensure: Quality of Service (QoS) The detection of bottlenecks The detection of attacks How to ensure these issues?  Statistics generated from existing log data Situation today 3 Does an AN have enough resources? Does it provide sufficient statistics at all?

4 60% Introduction & Motivation 4 CPU MEM Resources utilization

5 Simple support of new statistics types Simultaneous computation of multiple statistics Processing of increasing log data volumes Processor utilization RAM utilization Drops Number of packets Short term statistics (STS) for single ANs Supported Introduction & Motivation 5 Not supported Long term statistics (LTS) Creation of global statistics

6 One AN does not have enough hardware ressources  Usage of multiple ANs to compute statistics Efficient resource sharing with high resilience and scalability  Utilization of P2P technology DuDE: Exploitation of already available resources  No extra costs for additional equipment Introduction & Motivation 6

7 DuDE in General Logical P2P ring ID 7 Node1 Node2 Node4 Node3

8 DuDE in General 8 Log data (some hundreds of KBs) 8 Data chunk (ca. 100 KBs)

9 DuDE in General Objective: High log data availability = 99.999 % Simple replication wastes memory ressources  Reed-Solomon Codes Split log data of each AN into m data chunks 9

10 DuDE in General Objective: High log data availability = 99.999 % Simple replication wastes memory ressources  Reed-Solomon Codes Split log data of each AN into m data chunks Encoding: Add k interleaved coding chunks  n=m+k chunks 10

11 DuDE in General Objective: High log data availability = 99.999 % Simple replication wastes memory ressources  Reed-Solomon Codes Split log data of each AN into m data chunks Encoding: Add k interleaved coding chunks  n=m+k chunks Decoding: Restore log data from any m of n chunks 11

12 DuDE in General 12 Log data (some hundreds of KBs) Data chunk (ca. 100 KBs) How to apply our application to P2P?

13 Task Job = collection of STS and/or LTS tasks Task = part of job, e.g., request for „CPU“ statistics Jobscheduler (JS): Reception and monitoring of job Taskwatcher (TW): Reception and processing of task DuDE in General 13 Admin. Job Which steps are necessary to compute statistics?

14 60% 50% 30% 10% The DuDE Algorithm in Detail Stage 1: Resource collection 14 Admin. … Job … Task … Log data … Global statistics

15 The DuDE Algorithm in Detail 1. Resource collection 15 Admin. Stage 2: Jobscheduler determination … Job … Task … Log data … Global statistics

16 The DuDE Algorithm in Detail 16 Admin. 2. Jobscheduler determination Stage 3: Resource re-collection 1. Resource collection … Job … Task … Log data … Global statistics

17 The DuDE Algorithm in Detail 17 Admin. 3. Resource re-collection Stage 4: Task assignment 2. Jobscheduler determination 1. Resource collection Request for Processor utilization STS Request for RAM utilization LTS Request for Drops LTS … Job … Task … Log data … Global statistics

18 The DuDE Algorithm in Detail … Job … Task … Log data … Global statistics 18 Admin. 3. Resource re-collection Stage 4: Task assignment 2. Jobscheduler determination 1. Resource collection How to find all log data for global statistics computation?

19 Node1 Node2 Node3 Node5 no The DuDE Algorithm in Detail Request for global statistics  All data needed ID Taskwatcher iSucc?NN >=AID 1 2 3 4 5 yes no 0 0 0 1 2 yes Node1 Node2 Node3 Node4 Node5 Algorithm is done Global Peer Data Discovery Algorithm - Threshold value A = 2 Algorithm is done 5yes0no Node5 61no Node6 7yes2 Node7 19

20 The DuDE Algorithm in Detail 20 Admin. 4. Task assignment Stage 5: Log data collection 3. Resource re-collection 2. Jobscheduler determination 1. Resource collection … Job … Task … Log data … Global statistics

21 The DuDE Algorithm in Detail 21 Admin. 5. Log data collection 4. Task assignment 3. Resource re-collection 2. Jobscheduler determination 1. Resource collection Stage 6: Statistics computation Processor utilization stat. RAM utilization stat. Drops stat. … Job … Task … Log data … Global statistics

22 The DuDE Algorithm in Detail 22 Admin. 5. Log data collection 4. Task assignment 3. Resource re-collection 2. Jobscheduler determination 1. Resource collection Stage 7: Send results and display them 6. Statistics computation Admin. … Job … Task … Log data … Global statistics

23 The DuDE Algorithm in Detail 23 Admin. 5. Restore log data 4. Assign tasks 3. Resource recollection 2. Determine job scheduler 1. Resource collection Stage 7: Send results and display them 6. Compute statistics Admin. … Job … Task … Log data … Global statistics

24 Test Scenario and Evaluation 24 PC Configuration Pentium 4 (1.5 GHz) 512 MB RAM  Equivalent to AN HW

25 Test Scenario and Evaluation Parameters: Number of tasks inside job Number of log data sets in the P2P network Computational load for statistics computation Measurements: Time for finishing a job Memory utilization 25

26 Test Scenario and Evaluation 26 Linear Increase of Needed Time  Time is Constant

27 Test Scenario and Evaluation 27 Linear Increase of Needed Time  Time is Constant

28 Test Scenario and Evaluation Linear Increase of Memory Utilization  Constant Memory Utilization 28

29 Test Scenario and Evaluation Memory utilization increases more at the single AN than at the taskwatcher 29

30 Test Scenario and Evaluation Memory utilization is constant and independent of the computational load 30

31 Summary and Future work P2P-based system for distributed computing of statistics STS and LTS Statistics for a single AN and the whole network Global Peer Data Discovery Algorithm Successfully developed prototype (demo session) Investigation of further use cases 31

32 Thanks for your attention! Questions? 32

Download ppt "Of Rostock University DuDE: A D istributed Computing System u sing a D ecentralized P2P E nvironment The 4th International Workshop on Architectures, Services."

Similar presentations

Cultural Heritage in REGional NETworks REGNET Project Meeting Content Group 2002-06-24 - 2002-06-26.

Cultural Heritage in REGional NETworks REGNET Project Meeting Content Group
A P2P-based Storage Platform for Storing Session Data in Internet Access Networks T. Bahls, D. Duchow Nokia Siemens Networks Broadband Access Division.

A P2P-based Storage Platform for Storing Session Data in Internet Access Networks T. Bahls, D. Duchow Nokia Siemens Networks Broadband Access Division.
Muse confidential Broadband Europe 2007 We3A.4 Document:Emulation and Simulation Tool for Design and Optimization of IMS based FMC Networks Date:2007-12-05.

Muse confidential Broadband Europe 2007 We3A.4 Document:Emulation and Simulation Tool for Design and Optimization of IMS based FMC Networks Date:
Lecture 1: History of Operating System

Lecture 1: History of Operating System
Network Coding for Large Scale Content Distribution Christos Gkantsidis Georgia Institute of Technology Pablo Rodriguez Microsoft Research IEEE INFOCOM.

Network Coding for Large Scale Content Distribution Christos Gkantsidis Georgia Institute of Technology Pablo Rodriguez Microsoft Research IEEE INFOCOM.
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment Chapter 11: Monitoring Server Performance.

70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment Chapter 11: Monitoring Server Performance.
A Routing Control Platform for Managing IP Networks Jennifer Rexford Princeton University

A Routing Control Platform for Managing IP Networks Jennifer Rexford Princeton University
Hyper-Threading Neil Chakrabarty William May. 2 To Be Tackled Review of Threading Algorithms Hyper-Threading Concepts Hyper-Threading Architecture Advantages/Disadvantages.

Hyper-Threading Neil Chakrabarty William May. 2 To Be Tackled Review of Threading Algorithms Hyper-Threading Concepts Hyper-Threading Architecture Advantages/Disadvantages.
Implementation of Distributed Air Traffic Control Simulator Ranko Radovanović, Miloš Cvetanović, Zaharije Radivojević School of Electrical Engineering,

Implementation of Distributed Air Traffic Control Simulator Ranko Radovanović, Miloš Cvetanović, Zaharije Radivojević School of Electrical Engineering,
Mobility in the Virtual Office: A Document-Centric Workflow Approach Ralf Carbon, Gregor Johann, Thorsten Keuler, Dirk Muthig, Matthias Naab, Stefan Zilch.

Mobility in the Virtual Office: A Document-Centric Workflow Approach Ralf Carbon, Gregor Johann, Thorsten Keuler, Dirk Muthig, Matthias Naab, Stefan Zilch.
1 Energy Efficient Communication in Wireless Sensor Networks Yingyue Xu 8/14/2015.

1 Energy Efficient Communication in Wireless Sensor Networks Yingyue Xu 8/14/2015.
U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Black-box and Gray-box Strategies for Virtual Machine Migration Timothy Wood, Prashant.

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Black-box and Gray-box Strategies for Virtual Machine Migration Timothy Wood, Prashant.
Computer System Lifecycle Chapter 1. Introduction Computer System users, administrators, and designers are all interested in performance evaluation. Whether.

Computer System Lifecycle Chapter 1. Introduction Computer System users, administrators, and designers are all interested in performance evaluation. Whether.
Can Network Coding Help in P2P Networks? Dah Ming Chiu, Raymond W Yeung, Jiaqing Huang and Bin Fan Chinese University of Hong Kong Presented by Arjumand.

Can Network Coding Help in P2P Networks? Dah Ming Chiu, Raymond W Yeung, Jiaqing Huang and Bin Fan Chinese University of Hong Kong Presented by Arjumand.
PROOF: the Parallel ROOT Facility Scheduling and Load-balancing ACAT 2007 Jan Iwaszkiewicz ¹ ² Gerardo Ganis ¹ Fons Rademakers ¹ ¹ CERN PH/SFT ² University.

PROOF: the Parallel ROOT Facility Scheduling and Load-balancing ACAT 2007 Jan Iwaszkiewicz ¹ ² Gerardo Ganis ¹ Fons Rademakers ¹ ¹ CERN PH/SFT ² University.
Abstract Load balancing in the cloud computing environment has an important impact on the performance. Good load balancing makes cloud computing more.

Abstract Load balancing in the cloud computing environment has an important impact on the performance. Good load balancing makes cloud computing more.
Research on cloud computing application in the peer-to-peer based video-on-demand systems 2012.04.13 Speaker : 吳靖緯 MA0G0101 2011 3rd International Workshop.

Research on cloud computing application in the peer-to-peer based video-on-demand systems Speaker : 吳靖緯 MA0G rd International Workshop.
Introduction to HP LoadRunner Getting Familiar with LoadRunner >>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com

Introduction to HP LoadRunner Getting Familiar with LoadRunner >>>>>>>>>>>>>>>>>>>>>>
COLLABORATIVE EXECUTION ENVIRONMENT FOR HETEROGENEOUS PARALLEL SYSTEMS Aleksandar Ili´c, Leonel Sousa 2010 IEEE International Symposium on Parallel & Distributed.

COLLABORATIVE EXECUTION ENVIRONMENT FOR HETEROGENEOUS PARALLEL SYSTEMS Aleksandar Ili´c, Leonel Sousa 2010 IEEE International Symposium on Parallel & Distributed.
SEDA: An Architecture for Well-Conditioned, Scalable Internet Services

SEDA: An Architecture for Well-Conditioned, Scalable Internet Services

Similar presentations

Ads by Google