1. Yuriy Bulygin Security Center of Excellence Intel Corporation
CPU SIDE-CHANNELS VS. VIRTUALIZATION MALWARE: THE GOOD, THE BAD, OR THE UGLY
Yuriy Bulygin
Security Center of Excellence
Intel Corporation

2. AGENDA RSB based micro-architectural side-channel
Hyper-channel: detecting hypervisor with uArch side-channel
Demo
Conclusion
9/20/2018

3. RSB BASED μARCH SIDE-CHANNEL
9/20/2018

4. μARCH SIDE-CHANNELS Cache based side-channel attacks
(Simple) Branch Prediction Analysis (BPA)
Instruction cache analysis
Shared FU attack (shared multiplier in SMT capable CPU)
Crypto + Spy threads (software or hardware) share some CPU resource
Spy puts the shared resource in a known state and monitors if and how it was corrupted by crypto
Crypto may corrupt spy’s state depending on the secret (key)
Information about the secret leaks through this CPU resource and can be measured by the spy to recover the key
9/20/2018

5. RETURN STACK BUFFER (RSB)
Internal hardware “stack” in CPU
Typically simple push/pop stack structure with 16 entries
May be more complicated that simple stack on modern CPUs
Predicts target address of RET instruction before it’s available from memory
CALL instruction drives next linear IP (return address) into the RSB
target address of RET instruction is derived from the topmost RSB entry
RSB is circular buffer with respect to CALL’s: if RSB is full the oldest return address is overwritten
Mispredict penalty if it’s later determined that it doesn’t match return address popped from program stack
A.k.a. RAS (Return Address Stack)
9/20/2018

6. USING RSB TO SPY ON CRYPTO CODE
Spy thread executes 16 nested CALL instructions to fill RSB with spy’s return addresses
Crypto thread executes code (e.g. ER step in Montgomery modular reduction algorithm)
Spy thread then executes 16 RET instructions and measures time taken to execute them
Or directly measures “number of RSB misses” performance counter
Spy observes increased time due to RSB mispredictions corresponding to one or more spy’s return addresses replaced with crypto’s return addresses
What if crypto implementation replaced different # of RSB entries depending on key bit or result of mod multiplication ??
Spy would be able to differentiate key bit value based on # of RSB mispredictions
9/20/2018

7. FILLING RSB WITH SPY’S RET’urns
 crypto executes
RSB
Crypto thread executes square-and-multiply modular exponentiation or Montgomery modular multiplication (MMM)
Let’s take a look at this Montgomery reduction:
call func15 
call func14 
call func13 
call func12 
call func11 
call func10 
call func9 
call func8 
call func7 
call func6 
call func5 
call func4 
call func3 
call func2 
call func1 
call func0 
// Montgomery modular reduction
crypto_montgomery_reduction { ..
// End Reduction step
if( crypto_cmp(a, N) >= 0 ) {
crypto_sub(a, a, N); }
9/20/2018

8. CRYPTO CORRUPTS SPY’S RSB DEPENDING ON THE SECRET
No End Reduction (A < N)
if( crypto_cmp(a, N) >= 0 ) {
crypto_sub(a, a, N); }
The rest of spy’s return addresses
are not corrupted
End Reduction is carried out (A ≥ N)
if( crypto_cmp(a, N) >= 0 ) {
crypto_sub(a, a, N); }
crypto_sub replaces additional entries
9/20/2018

9. SPY OBSERVES RSB MISSPREDICTIONS
rdtsc
RSB
Spy can distinguish if crypto executed:
crypto_cmp only (1 RSB miss): MMM w/o End Reduction or
crypto_cmp/crypto_sub (4 RSB misses): MMM with ER step
ret ; func15 
ret ; func14 
ret ; func13 
ret ; func12 
ret ; func11 
ret ; func10 
ret ; func9 
ret ; func8 
ret ; func7 
ret ; func6 
ret ; func5 
ret ; func4 
ret ; func3 
ret ; func2 
ret ; func1 
ret ; func0 
RSB miss 
rdtsc
9/20/2018

10. HYPER-CHANNEL: USING RSB BASED μARCH SIDE-CHANNEL TO SPY ON HYPERVISOR
9/20/2018

11. OOPS. LET’S DO IT AGAIN
#VMEXIT  CPUID
RSB
Spy populates RSB by executing 16 nested CALL’s
Executes CPUID or any other instruction that causes #VMEXIT
If OS is in non-root (guest) mode then CPUID is trapped by hypervisor
call func15 
call func14 
call func13 
call func12 
call func11 
call func10 
call func9 
call func8 
call func7 
call func6 
call func5 
call func4 
call func3 
call func2 
call func1 
call func0 
9/20/2018

12. HYPERVISOR CORUPTS SPY RSB CONTENTS
#VMEXIT handler is likely to “corrupt” 1 or more spy’s RSB entries replacing them with its own entries
It enough for #VMEXIT handler to make 1 CALL to subfunction
13 hyper-channel return addresses
are not corrupted
vmexit_subfunc1: call vmexit_subfunc11 
vmexit_subfunc: call vmexit_subfunc1 
VMExit_Handler: call vmexit_subfunc 
9/20/2018

13. SPY OBSERVES RSB MISSPREDICTIONS
rdtsc
RSB
After #VMEXIT spy executes 16 RET’urns
RSB hit: < 3 clk cycles
RSB miss penalty: clk cycles
Experiment:
Clear: 83 cycles
Rootkit-ed: 123 cycles
Can be >300 cycles if #VMEXIT handler slightly modified
ret ; func15 
ret ; func14 
ret ; func13 
ret ; func12 
ret ; func11 
ret ; func10 
ret ; func9 
ret ; func8 
ret ; func7 
ret ; func6 
ret ; func5 
ret ; func4 
ret ; func3 
ret ; func2 
ret ; func1 
ret ; func0 
RSB miss 
rdtsc
9/20/2018

14. CLOSER LOOK AT THE RSB SPY ..
func15() {
cpuid ; #VMEXIT on VT
rdtsc ; start measurement
ret ; start 16 returns }
func14() {
call func15
ret ..
func0() {
call func1
spy() {
cli
call func0
rdtsc ; end measurement
sti }
9/20/2018

15. DEMO: HYPER-CHANNEL DETECTOR
9/20/2018

16. DEMO: HYPER-CHANNEL
9/20/2018

17. PROPERTIES
No false negatives !! A single RSB entry corruption is detectable
Hyper-channel needs to know time taken by 16 RET’s to execute on non-virtualized OS (noticed 100 in command-line ??)
“# of RSB misses” perf. counter is always 0 on non-virtualized OS !!
The RSB side-channel detection is probabilistic
RSB can be flushed due to multiple events
So the detector needs to make multiple measurements to decrease likehood of the false positive
Experimental probability of a false positive is ~ 1/1000 (RSB was flushed during hyper-channel’s measurement)
Make as few as 10 measurements
#VMEXIT behavior related to RSB depends on the core
RSB may be entirely flushed by #VMEXIT microcode
This is easily detectable but detector cannot tell anything about the hypervisor
Timing and TLB profiling are also side-channels
But there’s no externally published uArch side-channel using TLB’s
9/20/2018

18. EVADING HYPER-CHANNEL
Hypervisor may not make any calls inside VMExit handler
In this case hyper-channel detector will be useless
But this is a painful restriction !!
It’s similar to requiring crypto implementations to not make any key-dependant calls (what about recursive Karatsuba sqr/mul ??)
Clearly malicious hypervisor can masquerade legitimate VMM by making the same # of nested calls
It cannot evict all 16 entries as it’s suspicious !! Which legitimate VMM calls more than 16 nested subroutines ?? shoot it..
9/20/2018

19. CONCLUSION Side-channels are good..
Yeah, I know.. this conclusion sucks
Although many are tired of virtualization competition, let’s respect awesome research in virtualization rootkits and their detection
With widespread of HW virtualization, exploits targeting legitimate hypervisors may become as common as OS kernel exploits are now
We can detect that OS is virtualized, probably can detect malicious hypervisor by all known heuristics
So what ?? Can we remove it ??
9/20/2018

20. that uses embedded microcontroller in chipset
PLUG: DeepWatch
DeepWatch is a Proof of Concept hardware based detector of virtualization malware
that uses embedded microcontroller in chipset
to detect malicious hypervisor and remove it from the system
I hope you’ll see its demo soon..
9/20/2018

21. secure@intel.com http://www.intel.com/security
THANK YOU !! QUESTIONS ??
Thanks to researchers of virtualization rootkits, their detection methods, and uArch side-channel analysis
I’d also like to acknowledge Sagar Dalvi and Mark Davis from Intel
9/20/2018

23. REFERENCES Nate Lawson, Peter Ferrie, Thomas Ptacek:
Joanna Rutkowska, Alexander Tereshkin:
Dino A. Dai Zovi:
Peter Ferrie. Attacks on More Virtual Machine Emulators:
Edgar Barbosa:
Tal Garfinkel, Keith Adams, Andrew Warfield, Jason Franklin:
Michael Myers, Stephen Youndt:
bugcheck: vrdtsc
9/20/2018