Presentation is loading. Please wait.

Presentation is loading. Please wait.

Shinya Kitajima, Tetsuya Uchiumi, Shinji Kikuchi and Yasuhide Matsumoto System Management Lab., System Software Laboratories, FUJITSU LABORATORIES LTD.

Similar presentations


Presentation on theme: "Shinya Kitajima, Tetsuya Uchiumi, Shinji Kikuchi and Yasuhide Matsumoto System Management Lab., System Software Laboratories, FUJITSU LABORATORIES LTD."— Presentation transcript:

1 Shinya Kitajima, Tetsuya Uchiumi, Shinji Kikuchi and Yasuhide Matsumoto System Management Lab., System Software Laboratories, FUJITSU LABORATORIES LTD. Automatic Server Role Identification for Cloud Infrastructure Construction Copyright 2013 FUJITSU LABORATORIES LIMITED

2 Contents Background Misconfigurations in settings How to decide the same role servers? Our method Four rules Evaluation Accuracy rate Impact of four rules Conclusion Copyright 2013 FUJITSU LABORATORIES LIMITED1

3 Contents Background Misconfigurations in settings How to decide the same role servers? Our method Four rules Evaluation Accuracy rate Impact of four rules Conclusion Copyright 2013 FUJITSU LABORATORIES LIMITED2

4 Background Public cloud Resources and infrastructure are put together. [Resources] : CPU, Memory, Disk space... [Infrastructure] : Server, Switch, Network... Users rent the virtual resources. Copyright 2013 FUJITSU LABORATORIES LIMITED Virtual Resources Need to install more resources and infrastructure!! 3

5 Installation of new infrastructure Copy the configuration settings. Can reduce construction costs. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing infrastructureNew infrastructure Settings Operations manager Modify Network settings Have to modify network settings 4

6 Misconfigurations Sometimes misconfigurations occur. Servers cannot communicate with each other. The new infrastructure does not work properly. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing infrastructureNew infrastructure Settings Operations manager Modify Network settings Host name: manager2 IP address: DNS server: Gateway: Host name: manager1 IP address: DNS server: Gateway: Mistyping Forgot to change 5

7 Misconfiguration detection Our approach Detect the differences between the communication logs. New infrastructure is copy of existing infrastructure. Communication pattern should also be the same. [Communication log] Source IP address, source port, destination IP address, destination port Captured by tcpdump Copyright 2013 FUJITSU LABORATORIES LIMITED Server A Server B Server C Server A Server B Server C Existing infrastructureNew infrastructure Setting of Server B is wrong? 6

8 Two types of servers Management servers Organize cloud computing services User information Dom0 information Storage information Network information Charge information Dom0 servers Lent their resources as virtual resources to users e.g. CPU, memory, storage Settings of management servers are different from each other. Copyright 2013 FUJITSU LABORATORIES LIMITED Dom0 servers Management servers Focus only on management servers 7

9 Goal of our research Determine pairs of servers to compare the communication log. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing infrastructureNew infrastructure Can not compare the communication logs. ? 8

10 Goal of our research Determine pairs of servers to compare the communication log. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing infrastructureNew infrastructure Can compare the communication logs. 9

11 Goal of our research Determine pairs of servers to compare the communication log. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing infrastructureNew infrastructure DNS Mail NTP yum CMDB Same role server = Same communication logs 10

12 Applying Scene Copyright 2013 FUJITSU LABORATORIES LIMITED Plan Construc- tion Function Test Operation Test Operation Construction phase Operation phase Scene to apply our method Function test after construction Another, function test after update or improvement 11

13 Motivation You may think... However, A data center continues to change. The actual structure of data center changes from a plan gradually. New function, new machine, fix problems, etc. Constructers change a plan. They often do not install a new data center according to plan. Misconfigurations, unreasonable plan, etc. Copyright 2013 FUJITSU LABORATORIES LIMITED If there is a plan, it is not necessary to determine the same role server by using technique. If there is a plan, it is not necessary to determine the same role server by using technique. A plan is only a plan. 12

14 Difficulty of this problem It is difficult to know servers role from their appearances. The configurations of servers in both data center is not completely the same. Copyright 2013 FUJITSU LABORATORIES LIMITED Very similar What? Which? 13

15 Automatic identification To use communication logs is easy and make sense. We can also detect misconfigurations from communication logs. Communication logs influenced by misconfigurations. Consider the differences between the communication logs. Copyright 2013 FUJITSU LABORATORIES LIMITED Communication logs List of the same role servers Detect misconfigurations 14

16 Contents Background Misconfigurations in settings How to decide the same role servers? Our method Four rules Evaluation Accuracy rate Impact of four rules Conclusion Copyright 2013 FUJITSU LABORATORIES LIMITED15

17 Compare communication logs Summary of our method Copyright 2013 FUJITSU LABORATORIES LIMITED Assumption: configurations are almost the same. Compare communication logs Can observe almost the same communication logs. Unique Port Rule Corre- sponding Sources Rule Remaining Unique Port Rule Common Ports Rule Communication logs Our method (Four rules) Identification Existing data center New data center 16

18 Existing data center Basic idea of our method Same role server have the same listening ports Copyright 2013 FUJITSU LABORATORIES LIMITED Send packets NTP server Listening port for NTP Port number : 123 New data center Send packets NTP server IP > Listening port for NTP Port number : 123 NTP client NTP client IP > Communication log If the listening ports are the same, we can assume that those server have the same role. If the listening ports are the same, we can assume that those server have the same role. We call these servers as the corresponding servers. 17

19 Rule 1 : Unique port rule Focus on the unique listening port. Used by only one pair of servers. These pairs are the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Listening port number

20 Rule 1 : Unique port rule Focus on the unique listening port. Used by only one pair of servers. These pairs are the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Listening port number Unique listening port 19

21 Rule 1 : Unique port rule Focus on the unique listening port. Used by only one pair of servers. These pairs are the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number Corresponding servers 20

22 Rule 1 : Unique port rule Focus on the unique listening port. Used by only one pair of servers. These pairs are the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number Corresponding servers 21

23 Rule 2 : Corresponding sources rule Focus on the servers where the source servers are the corresponding servers. These pairs are the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number

24 Rule 2 : Corresponding sources rule Focus on the servers where the source servers are the corresponding servers. These pairs are the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number Corresponding servers Source servers are the corresponding servers 23

25 Rule 2 : Corresponding sources rule Focus on the servers where the source servers are the corresponding servers. These pairs are the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number Corresponding servers 24

26 Rule 3 : Remaining unique port rule Focus on the unique listening port again. Ignore the listening ports used by the corresponding servers. These pairs are the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number 25

27 Rule 3 : Remaining unique port rule Focus on the unique listening port again. Ignore the listening ports used by the corresponding server. These pairs are the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number

28 Rule 3 : Remaining unique port rule Focus on the unique listening port again. Ignore the listening ports used by the corresponding server. These pairs are the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number Corresponding servers Unique listening port 27

29 Rule 3 : Remaining unique port rule Focus on the unique listening port again. Ignore the listening ports used by the corresponding server. These pairs are the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number Corresponding servers 28

30 Rule 4 : Common ports rule Focus on the servers where the source servers are the corresponding servers again. Several servers are candidates for the corresponding servers. The servers that have most common listening ports as the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number

31 Rule 4 : Common ports rule Focus on the servers where the source servers are the corresponding servers again. Several servers are candidates for the corresponding servers. The servers that have most common listening ports as the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number Corresponding servers Source servers are the corresponding servers 30

32 Rule 4 : Common ports rule Copyright 2013 FUJITSU LABORATORIES LIMITED31

33 Rule 4 : Common ports rule Focus on the servers where the source servers are the corresponding servers again. Several servers are candidates for the corresponding servers. The servers that have most common listening ports as the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number

34 Rule 4 : Common ports rule Focus on the servers where the source servers are the corresponding servers again. Several servers are candidates for the corresponding servers. The servers that have most common listening ports as the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number Corresponding servers 33

35 Identification of remaining servers Apply the corresponding source rule again. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number

36 Identification of remaining servers Apply the corresponding source rule again. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number Corresponding servers 35

37 Identification of remaining servers Apply the remaining unique port rule again. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number

38 Identification of remaining servers Apply the remaining unique port rule again. Finally identify all servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center : Corresponding servers : Listening port number Corresponding servers 37

39 Contents Background Misconfigurations in settings How to decide the same role servers? Our method Four rules Evaluation Accuracy rate Impact of four rules Conclusion Copyright 2013 FUJITSU LABORATORIES LIMITED38

40 Evaluation environment Two small experimental cloud data centers Actual data center in our laboratory Management servers : 39 Dom0 servers Ignore the communication logs Recorded period One and a half day Enough to obtain almost all types of communication logs Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data centerNew data center Almost the same configuration 39

41 Evaluation contents Copyright 2013 FUJITSU LABORATORIES LIMITED Unique port rule Corresponding sources rule Remaining unique port rule Common ports rule 40

42 Evaluation result Results of accuracy rate Copyright 2013 FUJITSU LABORATORIES LIMITED Contribution to accuracy rate Repeatedly Applied Contain wrong answer Unique port ruleMiddle Corresponding sources ruleSmall Remaining unique port ruleHigh Common ports ruleMiddle 41

43 Conclusion Automatically identifies servers that have the same role. By comparing the communication logs. The accuracy rate is 94.1%. [Future works] Deal with the following cases The number of servers is different. The components working on servers is different. Propose a new misconfigurations detection method. By comparing communication logs. Use the corresponding servers list according to our method. Copyright 2013 FUJITSU LABORATORIES LIMITED42

44 Copyright 2013 FUJITSU LABORATORIES LIMITED43

45 Flow chart of our method Copyright 2013 FUJITSU LABORATORIES LIMITED START 1. Obtain communication logs 2. List ports used for communication 3. [Unique port rule] Identify the pairs using ports not used by other servers. 4. [Corresponding sources rule] Identify the pairs that have corresponding source servers. 5. [Remaining unique port rule] Identify the remaining pairs using unique ports. Is any server identified as the corresponding server? 6. [Common ports rule] Identify the pairs using most common ports. END No Yes Are all servers identified as corresponding servers? Yes No 44

46 Flow chart of our method Copyright 2013 FUJITSU LABORATORIES LIMITED START 1. Obtain communication logs 2. List ports used for communication 3. [Unique port rule] Identify the pairs using ports not used by other servers. 4. [Corresponding sources rule] Identify the pairs that have corresponding source servers. 5. [Remaining unique port rule] Identify the remaining pairs using unique ports. Is any server identified as the corresponding server? 6. [Common ports rule] Identify the pairs using most common ports. END No Yes Are all servers identified as corresponding servers? Yes No 45

47 Motivation Installation of new data center First, designers make a plan. Then, constructors install new data center. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data center Plan Additional function New equipment Improvement/Update New data center 46

48 Misconfiguration detection One of the promising approaches Detect the differences between the communication logs recorded in the existing infrastructure and the new infrastructure being developed. New infrastructure is copy of existing infrastructure. Communication pattern should also be the same. Copyright 2013 FUJITSU LABORATORIES LIMITED Server A Server B Server C Server A Server B Server C Existing infrastructureNew infrastructure We can assume that the setting of Server B is wrong? 47

49 How to define the same role servers? Manually identify the pair of servers that play the same role? Burdensome and time-consuming Sometimes cause discrepancies between the data center design and its actual implementations. The constructors of a data center and the designer of its infrastructure can be different persons. The infrastructure may often be updated to improve or add functions. Copyright 2013 FUJITSU LABORATORIES LIMITED We want to automatically identifies the pair of servers playing the same role in both infrastructures!! 48

50 Summary of our method Compare communication logs We can observe almost the same communication logs in which the servers play the same role. Copyright 2013 FUJITSU LABORATORIES LIMITED Existing data center New data center Packet capture Packet capture Communication logs Communication logs Analyze Corresponding servers Identify the same communication patterns. Assumption: configurations are almost the same. 49

51 Flow chart of our method Copyright 2013 FUJITSU LABORATORIES LIMITED START 1. Obtain communication logs 2. List ports used for communication 3. [Unique port rule] Identify the pairs using ports not used by other servers. 4. [Corresponding sources rule] Identify the pairs that have corresponding source servers. 5. [Remaining unique port rule] Identify the remaining pairs using unique ports. Is any server identified as the corresponding server? 6. [Common ports rule] Identify the pairs using most common ports. END No Yes Are all servers identified as corresponding servers? Yes No 50

52 Step 1 : Obtain communication logs Capturing packets We can use common tools such as tcpdump Existing data center New data center Capturing period Enough time to capture almost all types of packets To get earlier, we can run test programs or perform operations to generate communication. Record data IP address Source server Destination server Port number Source server Destination server Copyright 2013 FUJITSU LABORATORIES LIMITED Example of communication logs 51

53 Step 2 : List used ports for communication Analyze the port used for each communication List the source and destination servers and the listening port of each communication in the existing and new data centers. Copyright 2013 FUJITSU LABORATORIES LIMITED Example of list of used ports 52

54 Step 3: Apply the unique port rule Identify the servers that use a unique listening port. Find the unique port that is used as the listening port by only one pair of source and destination servers in each data center. Find the source and destination servers that use the same unique port in both data centers. Assume that these source servers and destination servers are the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED53

55 Step 3: Apply the unique port rule Identify the servers that use a unique listening port. Find the unique port that is used as the listening port by only one pair of source and destination servers in each data center. Find the source and destination servers that use the same unique port in both data centers. Assume that these source servers and destination servers are the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Source IP address Destination IP address Listening port of destination , , , Communication logs of existing data center Source IP address Destination IP address Listening port of destination , Communication logs of new data center 54

56 Step 3: Apply the unique port rule Identify the servers that use a unique listening port. Find the unique port that is used as the listening port by only one pair of source and destination servers in each data center. Find the source and destination servers that use the same unique port in both data centers. Assume that these source servers and destination servers are the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Source IP address Destination IP address Listening port of destination , , , Communication logs of existing data center Source IP address Destination IP address Listening port of destination , Communication logs of new data center 55

57 Step 3: Apply the unique port rule Identify the servers that use a unique listening port. Find the unique port that is used as the listening port by only one pair of source and destination servers in each data center. Find the source and destination servers that use the same unique port in both data centers. Assume that these source servers and destination servers are the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Source IP address Destination IP address Listening port of destination , , , Communication logs of existing data center Source IP address Destination IP address Listening port of destination , Communication logs of new data center 56

58 Step4 : Apply the corresponding sources rule Identify servers where the source servers are corresponding servers. Assume that a pair of servers (, ) in both data centers that satisfy the following conditions are the corresponding servers. The source servers (, ) of (, ) have been estimated as the corresponding servers. The destination servers of (, ) except for (, ) have also been estimated as the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED57

59 Step4 : Apply the corresponding sources rule Identify servers where the source servers are corresponding servers. Assume that a pair of servers (, ) in both data centers that satisfy the following conditions are the corresponding servers. The source servers (, ) of (, ) have been estimated as the corresponding servers. The destination servers of (, ) except for (, ) have also been estimated as the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED58

60 Step4 : Apply the corresponding sources rule Identify servers where the source servers are corresponding servers. Assume that a pair of servers (, ) in both data centers that satisfy the following conditions are the corresponding servers. The source servers (, ) of (, ) have been estimated as the corresponding servers. The destination servers of (, ) except for (, ) have also been estimated as the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED59

61 Step4 : Apply the corresponding sources rule Identify servers where the source servers are corresponding servers. Assume that a pair of servers (, ) in both data centers that satisfy the following conditions are the corresponding servers. The source servers (, ) of (, ) have been estimated as the corresponding servers. The destination servers of (, ) except for (, ) have also been estimated as the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Communication logs of existing data centerCommunication logs of new data center Source IP address Destination IP address Listening port of destination , , , Source IP address Destination IP address Listening port of destination ,

62 Step4 : Apply the corresponding sources rule Identify servers where the source servers are corresponding servers. Assume that a pair of servers (, ) in both data centers that satisfy the following conditions are the corresponding servers. The source servers (, ) of (, ) have been estimated as the corresponding servers. The destination servers of (, ) except for (, ) have also been estimated as the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Source IP address Destination IP address Listening port of destination , , , Communication logs of existing data center Source IP address Destination IP address Listening port of destination , Communication logs of new data center 61

63 Step 5 : Apply the remaining unique port rule Identify the servers that remain unidentified as corresponding servers that use the unique listening port for communication. Four servers (,,, ) are using the same port.,,, and are communicating with the port of,,, and respectively. (, ) and (, ) have already been estimated as the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED62

64 Step 5 : Apply the remaining unique port rule Identify the servers that remain unidentified as corresponding servers that use the unique listening port for communication. Four servers (,,, ) are using the same port.,,, and are communicating with the port of,,, and respectively. (, ) and (, ) have already been estimated as the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Source IP address Destination IP address Listening port of destination , , , Communication logs of existing data center Source IP address Destination IP address Listening port of destination , Communication logs of new data center 63

65 Step 5 : Apply the remaining unique port rule Identify the servers that remain unidentified as corresponding servers that use the unique listening port for communication. Four servers (,,, ) are using the same port.,,, and are communicating with the port of,,, and respectively. (, ) and (, ) have already been estimated as the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED Source IP address Destination IP address Listening port of destination , , , Communication logs of existing data center Source IP address Destination IP address Listening port of destination , Communication logs of new data center 64

66 Flow chart of our method Copyright 2013 FUJITSU LABORATORIES LIMITED We can identify the corresponding servers one by one. By applying the corresponding sources rule and the remaining unique port rule repeatedly. 65

67 Flow chart of our method Copyright 2013 FUJITSU LABORATORIES LIMITED We cannot identify all servers only by these rules. Use a port that is not unique for communication or that communicate with several servers whose corresponding servers are not yet identified. 66

68 Step 6 : Apply the common ports rule Identify servers where the source servers are the corresponding servers and that have common listening ports. We call such servers as candidate servers. A pair of servers having the largest number of common listening ports as the corresponding servers. Copyright 2013 FUJITSU LABORATORIES LIMITED67

69 Step 6 : Apply the common ports rule Copyright 2013 FUJITSU LABORATORIES LIMITED Server C : 25, 80, 443, 8080 Server D : 80, 123, 8080 Server C : 25, 80, 443, 8080 Server D : 80, 8080 Listening ports of each server CD C D Coincident rate 68

70 Step 6 : Apply the common ports rule Copyright 2013 FUJITSU LABORATORIES LIMITED Server C : 25, 80, 443, 8080 Server D : 80, 123, 8080 Server C : 25, 80, 443, 8080 Server D : 80, 8080 Listening ports of each server CD C D Coincident rate A pair of servers having the largest Co are the corresponding servers. 69

71 Evaluation contents Copyright 2013 FUJITSU LABORATORIES LIMITED70

72 Result detail Detail of the unique port rule Identified two types of servers as the corresponding servers. AP/DB type The application servers (APs) communicate with the database servers (DBs). The listening port for this communication differs from the database. Watch dog type Check whether the listening ports of other servers are alive or not. Communicate with the unique listening ports. Copyright 2013 FUJITSU LABORATORIES LIMITED unique port rule 71

73 Result detail Detail of the corresponding source rule Identified the servers that receive request messages (e.g. DNS, NTP) from many servers. The listening ports of these servers are unique. Multiple servers send request messages to these server. In our data centers, such servers are few. The impact of the corresponding source rule is small. Copyright 2013 FUJITSU LABORATORIES LIMITED Corresponding source rule 72

74 Result detail Copyright 2013 FUJITSU LABORATORIES LIMITED Remaining unique port rule 73

75 Result detail Copyright 2013 FUJITSU LABORATORIES LIMITED common ports rule 74

76 Result detail Detail of the unique port rule Identified two types of servers as the corresponding servers. AP/DB type The application servers (APs) communicate with the database servers (DBs). The listening port for this communication differs from the database. Watch dog type Check whether the listening ports of other servers are alive or not. Communicate with the unique listening ports. Copyright 2013 FUJITSU LABORATORIES LIMITED AP1 DB1 AP2DB2 AP/DB type Different Watch dog type Watch dog Alive or not? 75

77 Result detail Detail of the corresponding source rule Identified the servers that receive request messages (e.g. DNS, NTP) from many servers. The listening ports of these servers are unique. Multiple servers send request messages to these server. In our data centers, such servers are few. The impact of the corresponding source rule is small. Copyright 2013 FUJITSU LABORATORIES LIMITED e.g. DNS, NTP DSN, NTP 76

78 Result detail Copyright 2013 FUJITSU LABORATORIES LIMITED User certification Portal Certification Charge 77

79 Result detail Copyright 2013 FUJITSU LABORATORIES LIMITED78


Download ppt "Shinya Kitajima, Tetsuya Uchiumi, Shinji Kikuchi and Yasuhide Matsumoto System Management Lab., System Software Laboratories, FUJITSU LABORATORIES LTD."

Similar presentations


Ads by Google