1. Rootkits
Brent Boe
Vasanthanag Vasili

2. Rootkits: What is a Rootkit?
A rootkit is a set of tools used for (covertly) maintaining root access to a system
Rootkits allow attackers the ability to circumvent protection mechanisms limiting root access
Provide a much higher layer of stealth than normal “Trojan programs” by hiding processes and files

3. Rootkits: What is NOT a Rootkit?
A rootkit is not an exploit used to gain root access
Rootkits can only work if the attacker can gain administrative access
(Typical) Attacker sequence of events
Locate vulnerability on target host
Run exploit to gain root access
Install Rootkit
Remove Evidence
Locate next vulnerable host

4. Rootkit Functionality
Maintain Access
SSH (is for script kiddies)
Reverse shell (a bit unusual if servers initiate connections)
Covert channel backdoor – a signal system buried in an arbitrary field of a completely innocuous protocol.
Destroy evidence
Disable shell history (e.g. Linux - unset HISTFILE; export HISTFILE=/dev/null)
Kill syslog daemon and freeze the system log
Modify log files
Attack other systems
Local attack tools - Password Cracking, Capture root and access and obtain access to machines
Remote attack tools - Scanners and Autorooters
DOS tools –Conduct DOS attack on remote server
Clean the host system of previous infections
More than one rootkit can cause system instability and compromise the rootkit

5. What does a Rootkit hide?
The attacker’s files
The attacker’s processes (eg: sniffers, PW crackers)
The attacker’s user account
Unusual environment variables (network cards in promiscuous mode)
Specific network connections to and from compromised machines

6. Necessary Background User Space Kernel Space
The Kernel Space is more privileged than the User Space
The lower a rootkit can go, the more likely it is to avoid detection and defeat Host Intrusion Prevention Systems
User Space
Kernel Space

7. Necessary Background
The Intel x86 based chips use “rings” for access control with Ring 0 being the most permissive and Ring 3 being the most restrictive
User programs run in Ring 3
Kernel programs run in Ring 0
Rings 1 and 2 are unused
Ring 0

8. Types of Rootkits Binary Rootkits Kernel Rootkits System call Rootkits
Library Rootkits
Virtual Machine Rootkits
Database Rootkits
Runtime Kernel Patches
User Space
Kernel Space
Kernel Space and User Space

9. Binary Rootkits
These rootkits are collections of subverted popular system binaries (or executables).
Trojaned to perform action conducive to attacker (eg: hide malicious process)
Binary files usually precompiled for particular platform for user to choose & utilize correct one
Attacker deploys kit after breaking In via installation script which places binaries over original ones & saves old copies
On Linux, the attacker may choose to directly modify the source code on the target machine and recompile the binary.

10. Some trojaned binaries:
inetd, rlogin, rshd, sendmail, sshd, telnetd may contain magic password that provide access to attacker for remote access.
ps to hide processes from causal viewing by system admin.
netstat provides connection hiding
ls, dir provide file hiding
login,su,ping provide local access

11. Binary Rootkit Detection
Before the system is infected, compute the checksums of the binaries
CRC checksums
Cryptographic checksums
Better to store the checksums on separate media (i.e. CD-ROM) so an advanced attacker cannot modify the files
In practice, if a file (legitimately) changes frequently, this may lead to frequent checksum recomputations and false positives.
Checksum computation is used by the program Tripwire

12. Kernel Rootkits First reported in 1997
Loadable Kernel Modules hook into system kernel and modifies selected sys_call addresses stored in the system call table
Replaces the addresses of the legitimate sys_calls with the addresses of the sys_calls that are to be installed by the hacker’s LKM
Eg: KNARK ( targeting Linux2.2 Kernel)

13. Kernel Rootkits User Space Kernel Space LKM
Use Loadable Kernel Modules (LKMs) for Linux or Device Drivers for Windows
Full kernel access
User Space
Kernel Space
LKM

14. Kernel rootkit redirecting the system call table
Redirects the references to system call table
to new location.
New system call table is installed in new loc.
New system call table contains the address of malicious sys_call functons
Redirecting can be done by overwriting the pointer to the original system call table with the address of a new system call table that is created by the hacker

15. Kernel Rootkit Detection
Look for strange/inappropriate modules/device drivers
Keep in mind the binaries that would help examine this information may be compromised too.
/lib/modules
Prevent LKMs altogether by disallowing module loading
Sometimes a compile time option
StMichael
Monitors various portions of the kernel for modifications.
When “rootkit activity” is detected, attempts to restore to a previous good state

16. Necessary Background
When a process wants to communicate with the kernel it uses the system call table
The process throws a specific interrupt to pass control to the kernel
Windows – push the index of the system call in eax. Throw interrupt x2e
Linux – push the index of the system call into eax. Throw interrupt x80

17. System Call and Library Rootkits
Replaces the standard system library for relaying kernel information to a user process
The user library (libc) provides an interface to the system call table.
The advantage – no binaries need to change
Duplicate LKM functionality without entering the kernel space
Very easy to hide processes and files
T0rn8 kit most prominent one

18. System Call and Library Rootkit Detection
System calls like truss, strace, and ltrace can be used to trace the execution path of the system calls
Some integrity tools generate checksums against the system call tables.

19. Virtual Machine Based Rootkits (VMBR)
A VMBR moves the targeted system into virtual machine.
Instead of moving the attack code lower into the kernel space, it pushes the user higher into the user space
The previous (unhooked) OS runs over a virtual machine (as the guest software)
The guest is not allowed to interact with states outside of its Virtual Machine
The attacker has the liberty to run anything on the machine
Any anti-rootkit software run inside of the virtual machine will not detect any modifications to it’s state

20. Steps of VMBR installation
Modify the Boot Sequence to load the Virtual Machine Monitor (VMM) first
Modify it after shutdown after all monitoring processes have exited.
Interfere with the disk controller’s write – so that only the rootkit can store disk blocks
Working at this low level to avoid interference with monitoring software
Overwrite the master boot record so the VMBR loads first
Reboot and …
The target system is now running as a guest, you can interfere with them, but they can’t interfere with you

21. VM Rootkit Detection
Detecting a VM rootkit can be quite difficult (from the inside of the guest software)
Possible to detect a rootkit using instructions that reveal information about the kernel state (or the emulated kernel state)
redpill – uses the sidt instruction to store the interrupt descriptor table register. Since the VMM needs to move the emulated interrupt descriptor table, the ITDR will begin at a much higher address then it normally would.
Easiest way to detect a VM rootkit; boot from an alternate media.

22. Database Rootkits
A database can be considered a type of operating system
Users
Processes
Executables
Jobs
Symbolic Links

23. Database Rootkits 1st Generation Rootkits 2nd Generation Rootkits
Change the data dictionary (modify a view, procedure, and change synonyms)
For example, change ALL_USERS to be select * from sys.user$ u where u.name != ‘HACKER’;
2nd Generation Rootkits
Change the binary of the database so that all sys.user$ variables become sys.aser$
Remove the ‘Hacker’ entry from sys.user$
The system is now using sys.aser$ internally, but all integrity checks use sys.user$
3rd Generation Rootkits
For Oracle, Direct SGA (System Global Area) Manipulation – directly modify the contents of the database through modifying the memory the database is stored in

24. Database Rootkit Detection
Examine the internal views for obvious changes
Examine the internal system variables for any changes or new, unrecognized variables

25. Runtime Kernel Patching
Modifying the memory of the kernel while it resides in memory.
Simply modify a few bytes here, a FAR JMP there to execute the rootkit code, and you’re done.
A technique called detour patching totally that can totally circumvent executing code by modifying the control flow at runtime
Very difficult to detect
Very difficult to pull off successfully
Need extremely specific details about the target machine

26. General Rootkit Detection
Behavioral Detection
Look for suspect behaviors, such as writes to the memory containing important system call tables
Look for a change in the number, order, and frequency of calls
Signature Detection –search for unique byte patterns
Can be defeated through code obfuscation techniques
System Integrity Scans
Scan the kernel for inappropriate FAR JMP instructions
Detect unauthorized changes to loaded OS components in memory
Offline analysis of drives

27. Sony BMG Rootkit Scandal
Sony BMG Music Entertainment was sued in 2005 for surreptitious distribution of rootkit software on audio compact discs.
It used a software called Extended Copy Protection (XCP) designed to help prevent unlimited copying and unauthorized redistribution of the music on the disc.
XCP interferes with the normal way in which the Microsoft windows OS plays CDs
This causes the system vulnerable to malicious code
CD ROMS were inoperable due to the change in the registry settings caused by the software

28. Conclusion
Many rootkits practice “offense in depth,” and are by no means limited to only one of the techniques listed here.
Control of a system is determined by who can operate closer to hardware, or in the case of equal activity levels, who can best predict the actions of the other
The best way to fight rootkits is to prevent them from getting on your system in the first place – Intrusion Detection Systems, Host Intrusion Prevention Systems.

29. References
Beck, M et al. Linux Kernel Programming. 3rd ed. London: Addison Wesley, 2002.
Cesare, Silvio. “Runtime Kernel Patching.” 03 Mar 2007.
< >
Chuvakin, Anton. An Overview of Unix Rootkits. iDefense Labs: Feb 2003.
< >
Hoglund, Greg, Jamie Butler. Rootkits: Subverting the Windows Kernel. Addison
Wesley Professional: Upper Saddle River, NJ, 22 July  2005.
King, Samuel T. et al. SubVirt: Implementing malware with virtual machines. Mar 01
2007. < >
Kornbrust, Alexander. “Oracle Rootkits 2.0”. Black Hat 2006 USA, Las Vegas, NV. 02
Aug 06. < >

30. References
Levine, John G. et al. “A Methodology to Characterize Kernel Level Rootkits Exploits
that Overwrite the System Call Table”. IEEE < >
Locally checks for signs of a rootkit. 01 Mar Feb      <
Red-database-Security in the news/press. 23 Jan Red-Database-Security GmbH. 1 Mar     < >
Rootkit. 5 March Wikimedia Foundation Inc.26 Feb     <
Rootkits how to combat them   Kaspersky lab. 29 Feb     <
What is a rootkit? . 2 Mar    <
Zaytsev, Oleg. Rootkits, Spyware/Adware, Keyloggers and Backdoors: Detection and
Neutralization. A-List Publishing, Sep