1. Wolfgang von Rüden, CERN IT, August 20031 Computer Security: A permanent and costly battle Update for the CERN Management Board 26 August 2003 Wolfgang von Rüden CERN, IT Division Leader

2. Wolfgang von Rüden, CERN IT, August 20032 The VIRUS trend is going up ! Virus attacks have been with us for a long time and … They seem to get more and more sophisticated and destructive Our latest case is the Blaster Worm

3. Wolfgang von Rüden, CERN IT, August 20033 What happened ? Scandinavia's Nordea bank70 branch offices closed, worm in servers of all 440 offices Stanford University2,500 computers hacked (Blaster) CSX RailwaysCurtailed train service while restoring computer systems during 8 hours New York TimesAsked via public address system to shut off all computers (1/2 day) CFFWeb site problems for users (timetable, ticketing), long waits MarylandMotor vehicle administration affected Federal Reserve Atlanta/GABank affected Air Canada50% of phone reservation system capacity affected plus some check-in operations China2,000 intranet systems stopped OrsayWindows support on holidays

4. Wolfgang von Rüden, CERN IT, August 20034 Events since 16 July 2003 (1) 16 JulyMicrosoft releases a security bulletin warning about a so- called RPC vulnerability (MS03-26) affecting most versions of the Windows operating system 24 JulyIT launches a campaign to protect computers against this vulnerability. 5200 systems are patched (one command) 1 AugScan tool available: 500 vulnerable systems detected. Administrators contacted using Network DB information 11 AugustLeading antivirus companies warned about an exploit (W32.MSBlaster) rapidly spreading around the world. It is expected to make massive attacks against windowsupdate.com as of Saturday 16 August 13 AugustMail sent to each Division Leader with the list of vulnerable machines

5. Wolfgang von Rüden, CERN IT, August 20035 Events since 16 July 2003 (2) 15 AugustDespite multiple reminders, more than 200 Windows systems are still vulnerable. Site scanning shows suspicious activities, in particular via ACB or VPN, which are blocked for the week-end. Risk that those computers could launch the attacks and thereby potentially bringing down the whole or parts of the network and potentially reducing the ability of the organization to execute its mission. 18 AugustIT management decides to block vulnerable systems at the network level and to continue restrictions on the ACB and VPN service. No time to follow the usual consulting channels. Affected users are informed, provided the entry in the registration DB is up-to-date 18 AugustAn even more severe threat exploiting the vulnerability, "W32.Welchia", appeared and is now causing disruption at several sites 18 AugustTask force in place to help users to get back to normal. 19 AugustIn the afternoon, a mass mailing virus (W32.Sobif.F) started to appear at CERN and affects many users

6. Wolfgang von Rüden, CERN IT, August 20036 Status as of yesterday Better scanning tools in place Network tools added to block bad systems Still 150 systems blocked (half are portables and ~ 40 unregistered) More than 100 systems infected so far Both ACB and VPN are back, but restricted to common facilities (mail, web, file access) Helpdesk got 25% more calls More problems expected as people come back from holidays

7. Wolfgang von Rüden, CERN IT, August 20037 Initial problem analysis More than 500 machines not managed centrally caused the problems Some are CERN owned, locally managed machines and the owner did not follow the instructions (misunderstandings, manual action needed) Some machines are managed by the end-user or belong to visitors. In both cases, we have no means to enforce a patch “Black box” installations by companies Many insecure machines connected from home via ACB or VPN Major worry: Network DB often not updated by users, so they can even not be contacted.

8. Wolfgang von Rüden, CERN IT, August 20038 IT effort involved so far (FTE weeks) ActionPreventiveRepair Apply patch to 5000 machines via NICE 0.1 Security 4.0 Network group 6.0 User Support 3.5 Coordination 0.5 Local support 4.0 Total 0.118 Does not include effort in other Divisions The hotfix webpage was visited 12’200 times in August The emergency measures page 2600 times since 15 th August

9. Wolfgang von Rüden, CERN IT, August 20039 How can you help ? Insist in your Division to move as many machines as possible to the centrally managed service Nominate a security contact (and alternate) for your division to be contacted in case of alerts Independent machines must be managed by a person competent to apply patches and to ensure virus protection in compliance with OC5. Network DB must be updated whenever a computer moves or the owner changes Fast reaction time needed for security patches! “It’s on my list” is not enough Unmaintained “Black box approach” doesn’t work and should be banned. Secure your computer at home (or don’t connect)

10. Wolfgang von Rüden, CERN IT, August 200310 Proposed actions Enforce hardware address registration for all computers on site using DHCP (portable sockets and wireless) This will also apply to short-time visitors (i.e. FC delegates …) We are ready to start deploying this by the end of September, region by region, to be completed before Christmas Information campaign needed before enforcement ACB is a major security threat We need to move to another solution It also costs 500kCHF/year We propose to move to Internet Service Providers User pays local phone call or uses ADSL Need to understand impact on “poor” visitors Establish a “fire-fighting” procedure with short reaction time

11. Wolfgang von Rüden, CERN IT, August 200311 Conclusion CERN continued to work almost “as usual” while many other sites were knocked out Still, we need to be much more serious about security issues Top management has to buy in to achieve the goal We can’t afford a global break-down Thanks to my colleagues in IT who spent numerous hours overtime to keep things under control. Thanks also for all the help we got from the Divisions.

12. Wolfgang von Rüden, CERN IT, August 200312 Thank you ! Please help us protecting our work place