Presentation on theme: "Honeypots as a Tool to Improve Incident Response Readiness at USP"— Presentation transcript:
1 Honeypots as a Tool to Improve Incident Response Readiness at USP Alberto CamilliIsabel ChagasCentro de Computação EletrônicaUniversidade de São PauloEducause Security Professionals Conference 2007Denver12 April 2007Copyright CCE, Alberto Camilli, M. Isabel T. Chagas, 2007.This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.
2 Agenda University of São Paulo, numbers and IT organization USP and the national Honeynet projectUSP honeypot-based Early Notification procedure and resultsQ&A
3 Presentation Objectives Show how honeypots take part in the incident notification process, how they are configured and managed at USP.Main incident statistics for USP.Show what changes in the management of honeypots at USP lead to a change in the campus profile of incidents, with reduction in quantity and in the solution times.
4 University of São Paulo Research Extensive State University (São Paulo)185 in ISI rank:indexed paperscitations2.270 PhD, MScUndergraduate students4.884 facultystaffgraduate students8 main campuses, 60% based in São Paulo (city)85 “federation-like” Units (Teaching, Services and Extension)Annual budget near US$ 700 MUniversities devoted to research in Brazil are mostly offered by public institutions, either sponsored by the federal goverment or state government. There are also private institutions but normally they dont do research.The University of Sao Paulo is a state University, it is the one of the oldest and most traditional university respect diverse criterias, either geographic or respect to research,As we can seem from the numbers. There are 8 campuses in 5 different cities, mostly located in SP city.
5 Centro de Computação Eletrônica da USP Main USP Computer CenterNOC and CSIRT 24x7 operation400m2 data centerHPC main facilities for the University (e.g. computer 363o. in Top500 list)
6 USP Network Connections InternalGEANT( EUROPE / SPAIN )ExternalMGRJInternetCommodityUSA and BrazilSCPOP Clara( Global Crossing )RSInternet 2USAPOP RNP( USP )ANSPSPInternetUSPEnclaveKyateraTest BedRegistroBRMETRO IX( Non Commercial )GIGATest BedTVIn green we see the actual USP network backbone connectivity schema, in dark blue our connectivity to the federal network. We can also see our connection to Internet 2 and Geant.USPRibeirão PretoUSPnet600Mbps I/O Internet commodity traffic1,000 Buildings; 50,000 network points(80.000) accounts850,000 /day (20% valid)USPnetTelefonicaMetroSPMedicalSchoolsUSPBauruUSPPirassunungaUSPPiracicabaUSP( Cidade Universitária )USPSão CarlosHospitals
7 USP IT organization 143.107.xxx.xxx /16 200.136.xxx.xxx /20 DeanAdm.USP business rulesInternal UnitsAdministrationCIOIT services andinfrastructurecampus 1BusinessITCSIRTUSPInternal-ExternalCampus NotificationsIT in USP now broadly organized in two branches, one for the Administration of the day-by-day life of the University, like alumni registration and life-cycle, purchases, payment of salaries, and other network was only for academic purposes. The CIO’s role is to promote the development of infrastructure and teaching support initiatives having less influence on the IT related to business. Historically, for security concerns, two separate networks are available, with different IP range numbers, specially because administrative operations were of the client-server type, being the client computer a dumb terminal.The IP numeration were subnetted and distributed among the Units being simple to localize which Units are promoting a security incident by looking at its IP number.Now there is a tendency of these two IT structures become unified because with the advent of Web, the security is now at the application level and because unification of academic and administrative ID is in course.Notifications to/from the external world are done only by CSIRT USP. Incidents are then notified to the local Unites by the local CISIRTs which report back to CSIRT USP. In many cases CSIRT USP notifies directly Units in other campuses specially when an rapid incident solution is required. Initially we put honeypots only in the academic network which is a full class B.Recently we also put honeypots security in the the administrative network but they showed no attack coming from internal nework which is probably an inheritance of the old days’ security still being implemented.ID_#incLocalCSIRT1CampusComputerCentersCSIRTsNotificationsExternalcampus nLocalCSIRT2InternalCampus NotificationsID_#incID_#inc
8 USP and the Brazilian Honeypot Alliance Ribeirão PretoUSP PiracicabaUSPSão CarlosUSPCidade UniversitáriaCoordination by CERT.br:
9 The Project Brazilian Honeypots Alliance Distributed Honeypots Project Coordination: CERT.br and CenPRA Research CenterUse of low interation honeypotsBased on voluntary work of research partners37 research partner instituitionsIndustry, telcos, academic (USP and others), government and military networksEach partner providesHardware and networkHoneypot(s) maintenance
10 USP Motivation To increase USP’s capacity of: Incidents identification and knowledgeIncident detectionEvent correlation with other EntitiesTrend analysisWhich Units are more vulnerable?Very Useful for Incident ResponseSensors distributed in several campuses
12 Brazilian Honeypot Alliance Architecture Netblock range from /28 to /24
13 Data Usage Incident response (CERT.br): Partners: Identify well known malicious/abuse activitiesWorms, bots, scans, spams and malwares in generalNotify the Brazilian networks´contactsIncluding recovery tipsPartners:Observe trends and scans for new vulnerabilitiesDetect promptly:Outbreaks of new worms/botsCompromised serversNetwork configuration errorsWhat about USP usage?
14 Honeypots project: how good it was (july06) after 3 years? Gigabit backboneJuly 06Honeypot (CERT.br)ExternalInternal90%10%But ... how better can it become?
15 A closer look at our honeypot data... 1. Threats from the outsideDifferent External IPsPort 80Protect applications! Protect backbone routers!
16 A closer look at our honeypot data ... 2. Where and how the internal network is being attackedsame subnettype of attack, e.g.: 135(tcp), 445(tcp)Protect Windows desktop! Protect subnet!
17 Worm Propagation Times 300 min = 5 hsTypical log for a honeypot at USPWorm propagation Model ~ 10 hsthen ... early notification? Zou C., Gao L., Gong W. Monitoring and Early Warning for Internet Worms. CCS’03
18 Timeline logic in Early Notification Tp1TneTniTcaPropagationNotificationCert.brNotificationCSIRT-USPCorrectionActionTimeContamination (int)Improper Action (ext)CERT.brCSIRTHandlingCSIRTHandlingLocal CSIRTEnd ofIncidentTn = max(Tne,Tni)Tp2TcaTimeO propósito do HP é indentificar prematuramente a contaminação, minimizando o Tc+TnO propósito do CSIRT USP é minimimzar TnNotificationCERT.brORCSIRT-USPTp2 = Tp1 – some measured averageTca ??
19 Early Notification (EN) procedure HypothesisUnnoticed attacks should now begin to be identified.CSIRT-USP is able to notify attacks in advance.Units will be able to react accordingly to block these attacks.What we did?Notifiy the victims as soon as an internal attack is being observedNo further considerations about the nature of the attack.Why?We want better incident scores and honeypot logs are at our disposal.How and when?A daily script generates a summary of the attacks. Each attacked Unit receives the summary notification from CSIRT-USP, as a new security incident ticket.
20 Internal notifications message format CSIRT USP messages (daily summary):Subject: [Honeypot] Máquina(s) suspeita(s) ( )Content:zzz.yyy : 139(247)nnn.mmm : 135(202) 137(573) 139(3041) 1433(183) 445(1302) 80(568)Cert.br messages (on IP basis):Subject: zzz.yyy: host(s) infectado(s) com Agobot/PhatbotApr 27 13:30:zzz.yyy.3683 > xxx.xxx.xxx :S [tcp sum ok] (src OS: Windows XP SP1, Windows 2000 SP4) : (0) win <mss 1460,nop,nop,sackOK> (DF) (ttl 123, id 20447, len 48)number of attempts
21 Internal notifications results after 6 months of EN adoption Antecipated 6hsby CSIRT-USPSame data, Different analysis criteria,Different Interpretations
22 Overview of EN results CSIRT USP/Cert.BR internal notifications 2002-july 2006: internal notifications only from Cert.brNotificationsIncreaseExternalInternalBefore EN90%10%After EN50%
23 Classification of Incidents at USP DefinitionNotificationExpression orSymptomsCharacteristicsPossible Causes orAggravantsInternalCSIRT from USP alerting local Units worm/trojan in USP’s local nwCSIRT USPCert.brPort ScansBrute ForceCorrelated eventsSelf propagatingDifficult to correctDifficult to isolateExploring address spaceNon protected machinesUnpatched machines (mostly Windows)Infected machinesOpen ServicesExternalExternal entities complaining with USP about a problem in their nwExternal entitiesExternal CSIRTsP2PDefacementSpamOpen ProxiesOpen/RelaysPhishing/ScamIsolated EventsNot self propagatingEasy id of causesEasy correctionInadequate user behaviourBad configurationLong term contaminated machines
24 Top 10 Units solution time (before and after EN) Tca(days)Incident SolutionTime (avg)ΔT%Internal Incident Solution Time (avg)ΔTi%External Incident Solution Time (avg)ΔTe%01-0607-12(EN)ID_173109,4-6%6,510,866%10,37,2-30%ID_14616,811,4-32%13,411,3-16%16,911,6-31%ID_ 11220,415,5-24%11,215,236%22,212,6-43%ID_ 8612,415,9+28%7,08,217%12,514,7+18%ID_7018,516,1-13%4,616,2152%20,319,4-4%ID_557,6-14%5,8-44%6,8+6%ID_ 398,1-49%17,97,9-56%-38%ID_3917,27,8-55%15,38,21-46%17,6-59%ID_3814,44,9-65%10,05,714,61,8-88%ID_368,96,9-22%9,73,2-67%8,88,7-1%2006 PeriodUnit ID1, ,00 POLI2, ,00 CIRP3, ,00 FEA4, ,00 EESC5, ,00 IB6, ,00 FFLCH7, ,00 ECA8, ,00 FMRP9, ,00 IO10, ,00 CISC11, ,00 IF12, ,00 IGC13, ,00 IFSC14, ,00 IAG15, ,00 IME
25 Top 10 Units incidents (before and after EN) (2006)All IncidentsΔI %Internal IncidentsΔIi %External IncidentsΔIe%01-0607-12(EN)ID_17348115140%3812600%4534-24%ID_146697712 %5502500%6427-58%ID_11236114 %661220%3111-65%ID_ 862363174 %1252400%223873%ID_701555267 %403900%ID_551342223%429625%944%ID_ 3928155%2100%106-40%ID_392019-5%121100%7-63%ID_382117-19%2500%-74%ID_3612%8700%16-31%2006 PeriodUnit IDCorr(all,ext)=0,73Corr(int,ext)=0,22Corr(int1,ext1)=0,58Corr(ΔTe%, ΔIe%)=0,85POLICIRPFEA*EESCIB*FFLCHECAFMRPIOCISCIFIGCIFSCIAGIMEnow, a closer look to the profiles of incidents ...
26 Incident Profile ID_146 Before EN (69): After EN (77): F(t)=1-eλt 16%12%16%49%After EN (77):Tca ~ 20+ dayF(t)=1-eλt25%65%Tca ~12 day
27 Incident profile ID_112 Before EN (36): January-July 2006 After EN (77):July-December 20066%14%8%39%28%86%19%External (Spam, Open-Proxy, Other) “vanished”
28 EN limitations CSIRT USP Notification rate Local team internal OverloadedCapacity (λ´)Spare Capacity (λ)externalinternalF(t)=1-eλtinternalinternalexternalexternalinternalinternalIncident solutionrateFeedback to CSIRTLocal Responses are limited by local CapacitiesCapacity (skills, technology, staff/bw, staff/computers, ...)Local Capacities are related to the Local Incident Profiles (symptom)
29 Other (very) interesting profiles ... Similar Units profile (BW utilization, Staff, Technology, ....)ExternalInternalObs.ID_56542LinuxID_44413FWID_3ID_20146IFSC, IAG, FSP, IQLinux and good firewall managementMinimum contamination by wormsLittle interaction to CSIRT-USP, no influence from notification process.none from CSIRT USP !
30 Incident Response Readiness at USP Early notification is essentially a CSIRT procedure that relys on:Honeypots, for the localization and identification of the problemAvailable local internal capacities, for problem solvingLong term Incident reduction and better responses can be achieved with:EducationSpecializing local CSIRT managersTraining of local teams, to improve correction actionsGeneral User Education (especially on Windows): diversified public: students, professors, administrationPreventive actions, to keep volume of internal notifications under manageble limitsAnti-virus distributionBandwith controlNetwork access controlInstitutional network scanningOther Specific toolson-goingAutomática x ManualClassificação dos incidentesTarefa não trivialImportância da experiênciaNovo incidentes ou reincidência (qdo no mesmo IP)Resposta das UnidadesExternos respondidos antesMais fácil identificar a causaHP indicam problemas que requerem habilidades dos administradores das redes – falta de treinamento específicounder study
31 Summary Action taken by CSIRT-USP Possible Benefits Achieved results Notification by CSIRT USP (jul/06)Train CSIRT staffORCA-like monitoration with selectable featuresDeployment of more honeypots (jan/07)Antecipate treatment of local incidentImprove awareness and local treatmentIdentify local profiles and capacitiesAttack identification and analysysImprove CSIRT relationshipsBroader coverage of incidents notified (not obvious)Reduction in incident lifetime (not obvious)Reduction of external incidents (not obvious)Visual “Real-time” incident monitoring (feb/07)
32 ConclusionsEnd-user’s freedom is normally obtained with some degree of computers contamination.Honeypot is an effective way to detect early stages of contamination and to support the development of actions against later stages of the worm’s cycles.Honeypot monitoration is centralized and demands minumum infrastructure supportHoneypots permit suggest local actions according to Unit’s profilesGobal worm mitigation doesn’t necessarily mean local worm mitigation.Honeypot-based Early Notifications by CSIRT-USP changed the profile of security incidents at USPIncidents are closed in shorter timesExternal incidents has been reduced
33 Special Thanks CISRT USP TEAM Marta Bazzo Cilento Hamilton Jun HigashizonoAndré GerhardRogério Herrera MendonçaLuis FerreiraBruno DarigoFernando FugitaSolange VieiraOlavo Rodrigues
34 ReferencesBrazilian Honeypots Alliance – Distributed Honeypots ProjectCCECERT.brHoneydSeveral papers about the projectUSPOtherZou C., Gao L., Gong W.;Monitoring and Early Warning for Internet Worms. CCS’03Dagon D., Zou C., Lee W.; Modeling BotNet Propagation Using Time Zones. NDSS’06