Presentation on theme: "Honeypots as a Tool to Improve Incident Response Readiness at USP Alberto Camilli Isabel Chagas Centro de Computação Eletrônica Universidade de São Paulo."— Presentation transcript:
Honeypots as a Tool to Improve Incident Response Readiness at USP Alberto Camilli Isabel Chagas Centro de Computação Eletrônica Universidade de São Paulo Educause Security Professionals Conference 2007 Denver 12 April 2007
Agenda 1)University of São Paulo, numbers and IT organization 2)USP and the national Honeynet project 3)USP honeypot-based Early Notification procedure and results 4)Q&A
Presentation Objectives Show how honeypots take part in the incident notification process, how they are configured and managed at USP. Main incident statistics for USP. Show what changes in the management of honeypots at USP lead to a change in the campus profile of incidents, with reduction in quantity and in the solution times.
University of São Paulo Research Extensive State University ( São Paulo ) 185 in ISI rank: indexed papers citations PhD, MSc Undergraduate students faculty staff graduate students 8 main campuses, 60% based in São Paulo (city) 85 federation-like Units (Teaching, Services and Extension) Annual budget near US$ 700 M
C entro de C omputação E letrônica da USP Main USP Computer Center NOC and CSIRT 24x7 operation 400m 2 data center HPC main facilities for the University (e.g. computer 363 o. in Top500 list)
USP Network Connections USP Bauru USP Pirassununga USP Piracicaba USP São Carlos USP Ribeirão Preto Hospitals Medical Schools Telefonica Registro BR RJ MG SC RS SP Internet USP Enclave USP ( Cidade Universitária ) Kyatera Test Bed GIGA Test Bed METRO IX ( Non Commercial ) POP Clara ( Global Crossing ) GEANT ( EUROPE / SPAIN ) TV POP RNP ( USP ) USPnet Metro SP ANSP Internet Commodity USA and Brazil Internet 2 USA USPnet 600Mbps I/O Internet commodity traffic 1,000 Buildings; 50,000 network points (80.000) accounts 850,000 /day (20% valid) Internal External
USP IT organization Dean CIO Business IT ID_#inc CSIRT USP Local CSIRT1 Local CSIRT2 Administration ID_#inc CSIRTs Notifications Internal-External Campus Notifications Internal Campus Notifications Internal Units USP business rules Campus Computer Centers IT services and infrastructure campus 1 campus n External xxx.xxx / xxx.xxx / xxx.xxx /20 Adm.
USP Cidade Universitária USP Ribeirão Preto USP Piracicaba USP São Carlos Coordination by CERT.br: USP and the Brazilian Honeypot Alliance
The Project Brazilian Honeypots Alliance Distributed Honeypots Project Coordination: CERT.br and CenPRA Research Center Use of low interation honeypots Based on voluntary work of research partners –37 research partner instituitions Industry, telcos, academic (USP and others), government and military networks –Each partner provides Hardware and network Honeypot(s) maintenance
USP Motivation To increase USPs capacity of: –Incidents identification and knowledge –Incident detection –Event correlation with other Entities –Trend analysis –Which Units are more vulnerable? Very Useful for Incident Response Sensors distributed in several campuses
USP Honeypots Location
Brazilian Honeypot Alliance Architecture Netblock range from /28 to /24
Data Usage Incident response (CERT.br): –Identify well known malicious/abuse activities Worms, bots, scans, spams and malwares in general –Notify the Brazilian networks´contacts Including recovery tips Partners: –Observe trends and scans for new vulnerabilities –Detect promptly: Outbreaks of new worms/bots Compromised servers Network configuration errors What about USP usage?
Honeypots project: how good it was (july06) after 3 years? But... how better can it become? 3 years Gigabit backbone ExternalInternal 90%10% July 06 Honeypot (CERT.br)
A closer look at our honeypot data... Different External IPs Port 80 Protect applications! Protect backbone routers! 1. Threats from the outside
A closer look at our honeypot data... same subnet type of attack, e.g.: 135(tcp), 445(tcp) Protect Windows desktop! Protect subnet! 2. Where and how the internal network is being attacked
Worm Propagation Times 300 min = 5 hs ~ 10 hs  Zou C., Gao L., Gong W. Monitoring and Early Warning for Internet Worms. CCS03 Worm propagation Model  Typical log for a honeypot at USP then... early notification?
Timeline logic in Early Notification Notification Cert.br Notification CSIRT-USP End of Incident Time T ne T ca Contamination (int) Improper Action (ext) T p1 T ni Correction Action Propagation CERT.br CSIRT Handling CSIRT Handling Local CSIRT Notification CERT.br OR Notification CSIRT-USP T n = max (Tne,Tni) T p2 Tca T p2 = T p1 – some measured average Tca ?? Time
Early Notification (EN) procedure Hypothesis –Unnoticed attacks should now begin to be identified. –CSIRT-USP is able to notify attacks in advance. –Units will be able to react accordingly to block these attacks. What we did? –Notifiy the victims as soon as an internal attack is being observed –No further considerations about the nature of the attack. Why? –We want better incident scores and honeypot logs are at our disposal. How and when? –A daily script generates a summary of the attacks. Each attacked Unit receives the summary notification from CSIRT-USP, as a new security incident ticket.
Internal notifications message format CSIRT USP messages (daily summary): –Subject: [Honeypot] Máquina(s) suspeita(s) ( ) –Content: zzz.yyy : 139(247) nnn.mmm : 135(202) 137(573) 139(3041) 1433(183) 445(1302) 80(568) Cert.br messages (on IP basis): –Subject: zzz.yyy : host(s) infectado(s) com Agobot/Phatbot Content: Apr 27 13:30: zzz.yyy.3683 > xxx.xxx.xxx : S [tcp sum ok] (src OS: Windows XP SP1, Windows 2000 SP4) : (0) win (DF) (ttl 123, id 20447, len 48) number of attempts
Internal notifications results after 6 months of EN adoption Same data, Different analysis criteria,Different Interpretations Antecipated 6hs by CSIRT-USP
Overview of EN results CSIRT USP/Cert.BR internal notifications 2002-july 2006: internal notifications only from Cert.br Notifications Increase ExternalInternal Before EN90%10% After EN50%
Classification of Incidents at USP DefinitionNotificationExpression or Symptoms CharacteristicsPossible Causes or Aggravants Internal CSIRT from USP alerting local Units worm/trojan in USPs local nw CSIRT USP Cert.br Port Scans Brute Force Correlated events Self propagating Difficult to correct Difficult to isolate Exploring address space Non protected machines Unpatched machines (mostly Windows) Infected machines Open Services External External entities complaining with USP about a problem in their nw External entities Cert.br External CSIRTs P2P Defacement Spam Open Proxies Open/Relays Phishing/Scam Isolated Events Not self propagating Easy id of causes Easy correction Inadequate user behaviour Bad configuration Long term contaminated machines
Top 10 Units solution time (before and after EN) Tca (days) Incident Solution Time (avg) ΔT% Internal Incident Solution Time (avg) ΔTi% External Incident Solution Time (avg) ΔTe% (EN) (EN) (EN) ID_ ,4-6%6,510,866%10,37,2-30% ID_146 16,811,4-32%13,411,3-16%16,911,6-31% ID_ ,415,5-24%11,215,236%22,212,6-43% ID_ 86 12,415,9+28%7,08,217%12,514,7+18% ID_70 18,516,1-13%4,616,2152%20,319,4-4% ID_55 7,66,5-14%10,35,8-44%6,87,2+6% ID_ 39 15,98,1-49%17,97,9-56%15,29,4-38% ID_39 17,27,8-55%15,38,21-46%17,67,2-59% ID_38 14,44,9-65%10,05,7-43%14,61,8-88% ID_36 8,96,9-22%9,73,2-67%8,88,7-1% 2006 Period Unit ID
Incidents (2006) All IncidentsΔI %Internal Incidents ΔIi %External Incidents ΔIe% (EN) (EN) (EN) ID_ % % % ID_ % % % ID_ % % % ID_ % %223873% ID_ % %15 0 ID_ %429625%91344% ID_ % %106-40% ID_ % %197-63% ID_ %212500%195-74% ID_ %18700% % 2006 Period Unit ID Top 10 Units incidents (before and after EN) now, a closer look to the profiles of incidents...
Incident Profile ID_146 Before EN (69): After EN (77): Tca ~ 20+ day Tca ~12 day 49% 16% 12% 25% 65% F(t)=1-e λt
Incident profile ID_112 Before EN (36): January-July 2006 After EN (77): July-December 2006 External (Spam, Open-Proxy, Other) vanished 86% 28% 39% 14% 19% 8% 6%
EN limitations internal external internal external internal Notification rate internal Incident solution rate Spare Capacity (λ) Overloaded Capacity (λ´) Local Responses are limited by local Capacities Capacity (skills, technology, staff/bw, staff/computers,...) Local Capacities are related to the Local Incident Profiles (symptom) CSIRT USP Local team Feedback to CSIRT F(t)=1-e λt
Other (very) interesting profiles... Linux and good firewall management –Minimum contamination by worms –Little interaction to CSIRT-USP, no influence from notification process ExternalInternalObs. ID_56542Linux ID_44413FW ID_330FW ID_20146FW Similar Units profile (BW utilization, Staff, Technology,....) none from CSIRT USP !
Incident Response Readiness at USP Early notification is essentially a CSIRT procedure that relys on: –Honeypots, for the localization and identification of the problem –Available local internal capacities, for problem solving –Long term Incident reduction and better responses can be achieved with: Education –Specializing local CSIRT managers –Training of local teams, to improve correction actions –General User Education (especially on Windows): diversified public: students, professors, administration Preventive actions, to keep volume of internal notifications under manageble limits –Anti-virus distribution –Bandwith control –Network access control –Institutional network scanning –Other Specific tools under study on-going
Summary Action taken by CSIRT-USPPossible BenefitsAchieved results 1.Notification by CSIRT USP (jul/06) 2.Train CSIRT staff 3.ORCA-like monitoration with selectable features 4.Deployment of more honeypots (jan/07) 1. Antecipate treatment of local incident 2. Improve awareness and local treatment 3. Identify local profiles and capacities 4. Attack identification and analysys 5. Improve CSIRT relationships 1.Broader coverage of incidents notified ( not obvious ) 2.Reduction in incident lifetime ( not obvious ) 3.Reduction of external incidents ( not obvious ) 4.Visual Real-time incident monitoring (feb/07)
Conclusions End-users freedom is normally obtained with some degree of computers contamination. –Honeypot is an effective way to detect early stages of contamination and to support the development of actions against later stages of the worms cycles. –Honeypot monitoration is centralized and demands minumum infrastructure support –Honeypots permit suggest local actions according to Units profiles Gobal worm mitigation doesnt necessarily mean local worm mitigation. Honeypot-based Early Notifications by CSIRT-USP changed the profile of security incidents at USP –Incidents are closed in shorter times –External incidents has been reduced
Special Thanks CISRT USP TEAM –Marta Bazzo Cilento –Hamilton Jun Higashizono –André Gerhard –Rogério Herrera Mendonça –Luis Ferreira –Bruno Darigo –Fernando Fugita –Solange Vieira –Olavo Rodrigues
References Brazilian Honeypots Alliance – Distributed Honeypots Project CCE CERT.br Honeyd Several papers about the project USP Other Zou C., Gao L., Gong W.;Monitoring and Early Warning for Internet Worms. CCS03 Dagon D., Zou C., Lee W.; Modeling BotNet Propagation Using Time Zones. NDSS06