Project 2 Presentation & Demo Course: Distributed Systems By Pooja Singhal 11/22/2011 1

Outline Requirement Requirement Analysis Challenges Design Data Structures 3-Tiered Model Algorithms Implementation Learning and Experience Summary Acknowledgement Demo 2

Requirement Design and Implement a 3-Tiered Client Server Model. Given a City and State, find 5 nearest Airports. Using RPC System: Linux machine, Sun RPC, Language: C++/C 3

3) 3-Tiered Model Requirement Client Places Server Airport Server 4) Requirement of Algorithm To search Nearest Neighbors 4 Requirement Analysis 1) Parsing of Places2k.txt – Fast and Efficient. Which DS? 2) Parsing of airport-locations.txt - Spatial Partitioning. Which DS?

Challenges 3-Tiered Client Server Model. Spatial Partition: Did not know much about it! How to Search 5 nearest Airports ? Again No idea! 1) Parsing of Places2k.txt 2) Design and Implement Client 3) Design and Implement Places Server Test the code So far 4) Design and Implement 3-Tiered Model 5) Design and implement Spatial Partitioning of data in airport- locations.txt 6) Design and implement N nearest Airports Search 5

Design 6

Design Tactics Weekly Submissions made the job easy! First week Design: Parse both the files, Data Structures Second Week Design: IDL Design : Structure Design, Function Design Client Design and Logic: Places Server Design: Server for Client Final Week Design 3-Tiered Model Design Change of Data Structure for airport-locations.txt records Nearest Neighbor Search design 7

8 CLIENTPLACES SERVER AIRPORT SERVER 3-Tiered Model

Client Design Client Gets the location from User in the form of CITY and STATE. Pass it to Places Server Display the results 9

Places Server Design Phase 1 As a Server to Client: On start up, parse places2k.txt in hash table Hash Table key is combination of “CITY” and “STATE” Latitude and Longitude are stored as DATA along with the key Gets the inputs (CITY and STATE) from Client Make the Key: Apply Hash Function Search Hash Table If Found: Get Latitude and Longitude Phase 2: As a Client to Airport Server: Pass on Latitude and Longitude to Airport Server Get the result back from Airport Server Return the result to Client. 10

Airport Server Design 11 Act as a Server to Places Server On start up, parse airport-location.txt Creates a K-D Tree in memory Gets the latitude and location from places server. Search Nearest Neighbor Calculate the Distance Sort the results on Distance Return 5 neared neighbor back to places server

Design - Data Structures 12

Data Structures (1) Hash Table Store places2k.txt records Key is combination of CITY and STATE Data: Latitude and Longitude Advantages: Fast K-D Tree Spatial Partitioning of 2 Dimensional Points consist of Latitude and Longitude Node consists of 2 Dim points (Latitude, Longitude) Node Data: Airport Code, Airport Name, State Linked List Store Results consisting of 5 nearest airports Since, pointers do not get passed over RPC, needed to store the address of the next record 13

Data Structures (2) K-D Tree Space Partitioning Data Structure for storing a finite set of points in a k- dimensional space Invented by J Luis Bentley in 1975 Is a Binary Tree: Special example of Binary Space Partitioning Trees Applications in wide areas: Neural Networks, searching multidimensional data 14 Source: (2,3), (5,4), (9,6), (4,7), (8,1), (7,2) 1. X Plane Division 2. Y Plane Division 3. X Place Division

Design - Algorithms 15

Algorithms Nearest Neighbor Search Starting with the root node, the algorithm moves down the tree recursively Once the algorithm reaches a leaf node, it saves that node point as the "current best“ The algorithm unwinds the recursion of the tree, performing the following steps at each node: If the current node is closer than the current best, then it becomes the current best. The algorithm checks whether there could be any points on the other side of the splitting plane that are closer to the search point than the current best done by intersecting the splitting hyperplane with a hypersphere around the search point that has a radius equal to the current nearest distance.hyperplanehypersphere Since the hyperplanes are all axis-aligned this is implemented as a simple comparison to see whether the difference between the splitting coordinate of the search point and current node is less than the distance (overall coordinates) from the search point to the current best. If the hypersphere crosses the plane, there could be nearer points on the other side of the plane, so the algorithm must move down the other branch of the tree from the current node looking for closer points, following the same recursive process as the entire search. If the hypersphere doesn't intersect the splitting plane, then the algorithm continues walking up the tree, and the entire branch on the other side of that node is eliminated. When the algorithm finishes this process for the root node, then the search is complete. 16

17 Source:

18 Source:

19 Source:

20 Source:

21 Source:

22 Source:

Implementation 23

Implementation Weekly Submissions made the job easy! First week Implementation: Parsing of places2k.txt and airport-locations.txt, Storage in Hash Tables Second Week Implementation: Make Client and Places Server work Properly IDL implementation: client.x Implementation of Client Logic : client.c Places Server Implementation: places_server.c, client_svc.c Final Implementation Phase 1 : 3-Tiered Model Implementation Phase 2: Implementation of K-D Tree Phase 3: Implementation of Nearest Neighbor Search 24

Implementation Phase 1 : 3-Tiered Model Implementation 2 nd IDL : placesclient.x created places_server.c modified to call airport server Ist IDL client.x modified to include “host” input Phase 2 : K-D Tree Creation Airport server creates K-D tree and stores airport records. Libkdtree++ is used: kd_create(), kd_insert3() Phase 3: Nearest Neighbor Search kd_nearest_range(tree, point, radius) Calculation of Distance of all the points inside the circle with the Point Top 5 Nearest Airports were selected and returned Change of Resultant Structure: Made as Linked List 25

Learning and Experience GREAT LEARNING EXPERIENCE !! 3-Tiered Client - Server Model Data Distribution on different Servers Space Partitioning of multi dimensional data Search in Multi Dimensional data – Practical Approach Working with Hash Tables, K-D Trees, Linked List, Sort 26

Summary Implemented 3-Tiered Client Server Model Use of Hash Table to store places2k.txt Use of K-D Tree to store airport-locations.txt Use of Nearest Neighbor search algorithm Use of Linked List to return Result containing nearest airports 27

Acknowledgements Martin Krafft, Paul Harris, Sylvain Bougerel Library: libkdtree- Open Source STL Like implementation of K-D Trees 28

Demo 29

30