1. Paper Reading: Reporter: Shao-Yu Peng( 彭少瑜 ) Date: 2013/10/28

2. Outline Purpose Introduction Fluxing features of botnets Features detection techniques Comparison and evaluation Fluxing mitigation Future work Conclusion 2 /33

3. Purpose Summarized and classified the latest botnet fluxing features and detection techniques. Compared and Evaluated the surveyed techniques against multiple criteria. 3 /33

4. Introduction Botnet: A group of computers(bots/zombies) which controlled by the botmaster. In recent years, fluxing techniques have been applied to evade detection. 4 /33

5. Fluxing Features of botnet 5 /33

6. Fluxing features of botnets Fluxing methods are used to evade detected by hiding the domain-IP mappings. In our survey, we focus on two advanced mechanisms: 1. Fast flux(FF): a set of IP addresses-> a unique domain name 2. Domain flux(DF): a set of domain names-> a unique IP address 6 /33

7. Fast Fluxing, RRDNS and CDNs Ways to distribute loads of online services: 1. RRDNS(Round-robin DNS): Round-robin to response DNS requests. 2. CDNs(Content Distribution Networks): Computes the nearest servers to response. 3. Fast fluxing: Same idea but change entries more rapidly. Measuring and Detecting Fast-Flux Service Network Thorsten Holz 7 /33

8. Fast Fluxing Network Characters: Short TTLs, share one large IP pools…etc. Categories: 1. Single flux 2. Double flux 8 /33

9. http://www.honeynet.org/files/images/web-diagram.gif https://job.honeynet.org/files/images/dns-diagram.gif Fast Fluxing Network 9 /33

10. Domain Fluxing Network Server and bots generates domain names through same algorithm(consistently). Example: Torpig 10 /33

11. Current week, year Domain generation algorithm Current day Hard-coded domain names Configuration file failed Torpig:Bot failed Domain name 1 Domain name 2 Domain generation algorithm master success 11 /33

12. Features detection techniques Fast fluxing 12 /33

13. Detection techniques FF detection 1: Holz et al.: Distinguish btw normal network and fast fluxing network, and score a networks by: 1. #of IP-domain mappings in all DNS lookups, (more->higher prob. to be botnet) 2. #of nameserver records in one domain lookup, (more->higher prob. to be botnet) 3. #of autonomous system in all IP-domain pairs (more->higher prob. to be botnet) Limitation on detecting FFSN(benign) & FFAN(malicious) Measuring and Detecting Fast-Flux Service Networks 13 /33

14. Detection techniques FF detection 2: Zhou et al.: 1. To speed up Holz method 2. Improvement speed by combining results: (1) From different DNS servers; Build and share one suspicious IP address list. (2) From different suspect FF domains. Compare responses from domains to speed up confirmation. Collaborative Detection of Fast-Flux Phishing Domains 14 /33

15. (1) Server 1 Server 2 Server 3 Switch Address blacklist Each server: List’ = List 1 ∪ List2 ∪ List3 (2) Server FF domain 1 FF domain 2 FF domain 3 Unknown domain List’= Response 1 ∪ Response 2 ∪ Response 3 Response 1 Response 2 Response 3 Response 4 15 /33

16. Detection techniques FF detection 3: Caglayan et al.: 1. Monitor the DNS of a website by minutes. 2. Sensors, FF monitor/database, FFM classifier 3. Sensors monitor parameters including TTL…etc. and store into database. 4. Classifier evaluate a website with the analytic data in database. Real-time detection of as flux service networks 16 /33

17. Sensor FF monitors FF domain FFM database Unknown domain FF domain Classifier Unknown Website with rapidly changed IP 17 /33

18. Detection techniques FF detection 4: Perdisci et al.: Detect malicious ones from FFSN. 1. Monitoring FFSN traffic with a pre-filter by four features: (1) Short TTL, (2) The change rate of the set of resolved IPs returned, (3) A large number of resolved IPs, (4) Resolved IPs scattered across different networks. 2. Clustered domains with high relations 3. Classified domains according to the resolved IP address 4. Build a network classifier based on above data. FFSN=Fast-flux service network FFAN=Fast-flux attack network Detecting malicious flux service networks through passive analysis of recursive DNS traces 18 /33

19. Detection techniques FF detection 5: Yu et al. Distinguish FFSN and FFAN by agent lifespan. 1. Send request once per hour during 24 hours. 2. FFSN: 24/7 available; FFAN: unpredictable. 3. AOR(average online rate/24 hours) 4. MAR(minimum available rate/history record) 5. Detector judges btw FFAN and FFSN by AOR and MAR record by monitors. Fast-flux attack network identification based on agent lifespan 19 /33

20. Features detection techniques Domain fluxing 20 /33

21. DF detection 1: Stone-Gross et al.: 1. To determine the size of a botnet 2. Research on real world botnet –Torpig 3. Register the.com and.net domain which would be used by the botnet. 4. Log requests and record network traffic. 5. Determine the size by counting unique nodes. Your botnet is my botnet: analysis of a botnet takeover Detection techniques 21 /33

22. Detection techniques DF detection 2: Ma et al.: Distinguish domain fluxing network and normal network. 1. URL analysis based. 2. Lexical features and host-based features (1) Lexical: URL length, #of dots in URL, bag-of-words…etc. (2) Host-based: IP, domain name, location, connection speed… 3. Independent of content and structure. 4. Combination of all features -> highest accuracy. Beyond blacklists: learning to detect malicious web sites from suspicious URLs 22 /33

23. Detection techniques DF detection 3: Jiang et al.: Distinguish domain fluxing network and normal network, and classified. 1. Failed DNS queries come mainly from malicious activities. 2. DNS failure graph (bots with same DGA will create dense failure graph) 4. Analyze the graph structure and refer to domain name blacklists. Identifying suspicious activities through DNS failure graph analysis 23 /33

24. Detection techniques DF detection 4: Prakash et al.: Evaluation based on blacklists. Since Black listing method needed to exactly match URL, it is easy to evade. Model: Score new URL against an existing blacklist with 5 heuristics: 1. Replace the top-level domains 2. IP address equivalence (Same IP->change dir/path) 3. Directory structure similarity (different IP, similar path-> change filename) 4. Query string substitution (Same structure->change query) 5. brand name equivalence (3) ex: www.abc.com/online/singin/ebay.htm www.xyz.com/online/singin/paypal.htm Change filename-> www.abc.com/online/singin/paypal.htm www.xyz.com/online/singin/ebay.htm (4) ex: www.abc.com/online/singin/ebay?XYZ www.xyz.com/online/singin/paypal?ABC Change query-> www.abc.com/online/singin/ebay?ABC www.xyz.com/online/singin/paypal?XYZ Phishnet: Predictive blacklisting to detect phishing attacks (5) ex: www.abc.com/online/singin/ebay.htm Change brand name-> www.abc.com/online/singng/yahoo.htm 24 /33

25. Detection techniques DF detection 5: Yadav et al. Distinguish DF domain names from normal domain names. 1. Identify domain names generated by algorithm by spelling or pronounceable features. 2. Group DNS queries by TLD/IP-address 3. For each group, use Jaccard index to characterize alphanumeric distribution. Detecting algorithmically generated malicious domain names 25 /33

26. Database of non-malicious bigrams Suspicious URL, ex: ickoxjsov.botnet.com Ic,ck,ko,ox,xj,js,so,ov Subset with 75% of bigrams ex: the quick brown fox jump sover the lazy dog Calculate JI = (A∩B)/(A ∪ B) ex: 6/(8+35-6) = 0.16 Break into bigrams Average JI 26 /33

27. Comparison between techniques 27 /33

28. Comparison between techniques FF: 5 criteria: 1. Real-time 2. Accuracy 3. Distinguish FFSN VS. FFAN 4. Speed 5. Mining based Above these criteria, Is this meaningful to compare the algorithms with different goals? DF: 4 criteria: 1. Accuracy 2. Speed 3. Passive or active 4. Mining based 28 /33

29. dash line: not discussed or unclear in a paper A Survey on Latest Botnet Attack and Defend 29 /33

30. Fluxing Mitigation Need collaboration of both registers and ISPs. Blacklisting-related method is almost the only way. 30 /33

31. Future directions Data mining can be used widely to extract features. Graph spectra can be employed to study botnets. How to get the trust of remote owners which has compromised computers. Predict botnet writers new developed strategies. 31 /33

32. Conclusion Advantages: Survey on latest fluxing detection techniques of botnet. Drawbacks: The meaning of comparison btw algorithms with different purposes is vague. 32 /33

33. Thank you for listening