Cloud computing for internet emulator

Professor Muthucumaru Maheswaran Team Members Mia Hochar Simon Foucher David El Achkar David El Achkar Marc Atie Marc Atie GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Professor Muthucumaru Maheswaran Mia Hochar Simon Foucher David El Achkar David El Achkar Marc Atie Marc Atie D. Database Recomendations C. GINI Alterations Design Dilemma A. Evolution of GINI B. Dispatcher Objectives GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie E. Scheduler F. Conclusion

1. What is GINI? 2. GINI’s Features 3. Improvements Overview GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

GINI Defined What is GINI? Toolkit for creating virtual micro Internets Create midsize networks Process creates virtual instances of network elements Purpose: Teaching and learning tool Suitable to many levels of knowledge Future applications GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

GINI Snapshot GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Features Simple to use GUI Fully open-source system Standard compliant router Ability to implement additional protocols or new network elements GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Improvements Overview GINI on a Cloud © March ClientWorker Front EndBack End David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Improvements Overview GINI on a Cloud © March ClientWorker Front EndBack End David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Improvements Overview GINI on a Cloud © March Front EndDispatcher Database Scheduler Back End ServerWorkersClient David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Drawbacks Complicated Installation process GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Drawbacks OS Compatibility issues GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

GINI Components: ImprovementsBasics Database Scheduler Dispatcher gRouter GiniLinux uSwitch WGINI gLoader gBuilder GINI’s Components gBuilder gLoader gRouter GiniLinux uSwitch WGINI GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

1. Overview 2. Dispatcher Request 3. Connection Creation 4. Dispatcher Logging GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Overview Request/Reply Front EndDispatcher Database I need a worker Worker’s DNS Lowest CPU Usage GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Dispatcher Request Request/Reply Dispatcher DNSIPuptime Lab Lab Lab ……… Return worker DNS extractWorkerFromDB() GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Connection Creation Connection Established GINI on a Cloud © March Front EndBack End SSH Tunnel David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Dispatcher Logging Recording Connection Dispatcher Database 1.Send: Client IP Worker DNS 2.Connection Recorded: time stamp GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

1. Design Paradigm 2. Connection to Server 3. Design Dilemma GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Design Paradigm Transparent to user Component add-on rather than alteration GINI remains independent Only altered the Configuration file GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Design Paradigm Use of flexible communication library: twisted GINI already in Python Flexible Available source code Stable High-level functions GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Current Access to Remote Server GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Connection to Server 23 GINI on a Cloud © March 2010 David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Connection to server 24 GINI on a Cloud © March 2010 David El Achkar, Simon Foucher, Mia Hochar, Marc Atie Front EndBack End SSH Tunnel

Addition to GINI GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Addition to GINI GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Gateway to the cloud GINI on a Cloud © November 0927

Sequence of Events 1. Send dummy variable Front EndDispatcher Database NoWorker 28 GINI on a Cloud © March 2010 David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

1. Send dummy variable Sequence of Events 2. Dispatcher searches for a worker Front EndDispatcher Database 29 GINI on a Cloud © March 2010 David El Achkar, Simon Foucher, Mia Hochar, Marc Atie Best Worker?

1. Send dummy variable 2. Dispatcher searches for a worker 3. Returns worker or dummy variable Sequence of Events Front EndDispatcher Database 30 GINI on a Cloud © March 2010 David El Achkar, Simon Foucher, Mia Hochar, Marc Atie Worker’s DNS name

Result: Unaltered GINI 31 GINI on a Cloud © March 2010 David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Connection to server 32 Connection remains unaltered User needs no knowledge of Back End location Front EndBack End SSH Tunnel GINI on a Cloud © March 2010 David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Purpose of Dummy Variable Need to establish TCP communication Recycled as an error flag Could be used to convey info in future upgrade GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Flexibility Non-fixed data types being passed Currently a string: IP, DNS or anything else 34 GINI on a Cloud © March 2010 David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Drawback Temporary TCP connection: Port expiration at timeout Most Clients don’t have DNS 35 GINI on a Cloud © March 2010 David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Design Dilemma 36 Maintain permanent client-dispatcher connection Resources consuming Might not bring any tangible benefits Requires alterations to gServer and core Config Requires 3 communication events (instead of 1) Drop the connection after delivering the goods Lightweight component No extra resources used Impossible (or difficult) to retrieve clients GINI on a Cloud © March 2010 David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

1. Contents 2. Dispatcher Database Relationship 3. Design choices 4. Recommendations GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

DB keeps history of 10 last CPU usages for every worker Initial Design Database Storage: workers Table WorkersUsageHistory W134% … W249% … W357% … ……… Database GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Database Storage: workers Table Uptime is the average load on the system processor workerIDdnsNameuptime 1lab lab lab ……… Database GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie Implementation

Database Storage: clients Table DB keeps track of all the connections and their start time Database GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie ClientConnected toTime Stamp : : :56 ……… Initial Design

Database Storage: clients Table Time stamp is calculated in minutes starting midnight Database GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie clientIDworkerNameIP addresstimeStamp 1Lab Lab Lab ………… Implementation

Database - Dispatcher GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie Database Dispatcher workerIDdnsNameuptime 1lab lab lab getFreeWorker() 1 Lab Sort workers table by uptime 2

Database - Dispatcher GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie Database Dispatcher clientIDworkerNameIP addresstimeStamp 1lab lab lab lab Connection between client and lab2-39 4

Client Log GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie Call to the function displayClientsLog()

Design Choices Database implemented in SQL Used MySQLdb: wrapper around MySQL Compatible with Python API Database stored on the SOCS MySQL server Free, available and provided by McGill Database Name: 2010GINIdb Machine Name: mysql.cs.mcgill.ca Database can be reset at anytime GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Managing Client Connections Killing a connection Polling (time out): Dispatcher periodically polls client No response: dispatcher terminates connection Manual Client shuts down connection Sends message to dispatcher Too complex to implement: Needs modification to the core GINI code GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Managing Client Connections Database functions provided: getTimedOutClients() updateClientTime() removeClientByID() removeClientByIP() GINI on a Cloud © November 0947 David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

1. Purpose 2. Design Choices 3. Methodology 4. Sample Output 5. Recommendations GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Purpose Manage workers and their connections Examine CPU usage Ensure all workers are accessible GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Design choices GINI on a Cloud © March Paramiko Module for Python Implements secure SSH protocol Uptime Load average for past 1, 5, and 15 minutes No need to store each worker’s history David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Methodology GINI on a Cloud © March Retrieve workers list from database Get CPU usage of workers SSH connection to each worker Get “uptime” information Update database entry Sleep for one minute, then start again David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Database Update GINI on a Cloud © March Database Scheduler lab2-42 New uptime workerIDdnsNameuptime 1lab lab ……… 4 Request Workers List 1 lab2-42, lab2-39, … 2 lab2-39 New uptime 0.12 David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Sample Output GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Dynamic Update GINI on a Cloud © March Scheduler Worker $ gAddWorker<> $ gRemoveWorker<> David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Dynamic Update Two Bash scripts on server: addWorker.sh removeWorker.sh Boot configuration and shut down sequence need to be updated. $ /removeworker.sh $ /addworker.sh GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Workers Computers in the Krieble lab All machines could be shut down simultaneously No available workers Worker could be disconnected Workers Log could be used to find new worker Initiate new connection automatically GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Configuration Change passwords: SSH access to SOCS server Database access to SOCS MySql server GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

1. Summary 2. Advantages GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Summary Initiating a connection ClientDispatcher Database Worker 1. Client contacts dispatcher 3. Dispatcher sends IP address of worker to the client 2. Dispatcher queries database for worker, updates database accordingly 4. Client connects to worker through SSH tunnel GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Advantages No need to install back end Easier installation (less packages) Saves user trouble of software maintenance / update Cross-platform compatibility Back end can only run on Linux Front end supported by Linux and Windows GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Advantages Mobility Back end runs on remote server Users can run GINI from wherever they are Resource usage Managed, automated distribution of computing resources among users Easy to monitor and to add more workers GINI on a Cloud © March David El Achkar, Simon Foucher, Mia Hochar, Marc Atie

Questions? GINI on a Cloud © March