Copyright © 2015 Miao Yu, Virgil D. Gligor, and Zongwei Zhou CyLab and ECE Department Carnegie Mellon University {miaoy1, ACM CCS Denver, Colorado October 14, 2015 Trusted Display on Untrusted Commodity Platforms 1

Copyright © Picture: GEEK.COM. Insensitive Application (App) Insensitive Application (App) Sensitive Application (SecApp) Sensitive Application (SecApp) Sensitive Application (SecApp) Sensitive Application (SecApp) Sensitive Application (SecApp) Sensitive Application (SecApp) Security: no malicious scrapping/painting of SecApps output on Shared Displays Secure Display Sharing

Copyright ©  Security while maintaining: Sec- App 1 Operating System (unmodified) App Graphics Processing Unit (GPU) … Sec- App 2 App SecApp  User Perception Ideal Trusted Display  Compatibility Trusted Computing Base  Assurance Graphics Processing Unit (GPU)

Copyright © 2015  Security while maintaining:  Compatibility  Assurance  User Perception App SecApp Sec- App 1 Operating System (unmodified) App Graphics Processing Unit (GPU) … Sec- App 2 Commodity OS X GPU Managed by: Related Work Full Virtualization Hypervisor Full Virtualization Hypervisor X X ✓ ✓ Graphics Processing Unit (GPU) TCB X X Trusted Computing Base (TCB) Graphics Processing Unit (GPU)

Copyright © GPU Instructions Local Page Tables CPU Programs (e.g., drivers, Apps) Data (e.g., frame buffers) GPU Address Spaces  Objects Global Page Table (GGTT) Config. Registers Commands Background: GPU

Copyright © GPU Config. Registers CommandsInstructions Local Page Tables Display Engine Processing Engine CPU Programs (e.g., drivers, Apps) Other Engines GPU Address Spaces  Objects  Engines Global Page Table (GGTT) Data (e.g., frame buffers) Background: GPU

Copyright ©  Multiplexes GPU among VMs => Access mediation & emulation for GPU objects, e.g. GPU configuration registers  Reduces complexity => “address space ballooning” * Derived from Figure 7 of Tian et al. “A Full GPU Virtualization Solution with Mediated Pass-Through” Background: Full GPU Virtualization VM 2VM 1 GPU Global Page Table (GGTT) Ballooned

Copyright © VM 2VM 1 GPU Global Page Table (GGTT) * Derived from Figure 7 of Tian et al. “A Full GPU Virtualization Solution with Mediated Pass-Through” Ballooned  Multiplexes GPU among VMs => Access mediation & emulation for GPU objects, e.g. GPU configuration registers  Reduces complexity => “address space ballooning” => non-contiguous GPU address space Background: Full GPU Virtualization

Copyright ©  GPU instructions could be malicious => base & bound registers High Base Bound VM2 VM1 Low Base Bound High GGTT VM1 VM2 VM1 VM2  Inadequate GPU HW - single register pair for non-contiguous address spaces Insecurity of Full GPU Virtualization

Copyright ©  Insecure: Inadequate GPU HW - malicious GPU instructions break GPU address space separation  Lacks assurance: unverifiable code base - multiplexing GPU among VMs is complex e.g., emulating accesses to all GPU configuration registers Full GPU Virtualization In Summary Trusted Computing Base  Incompatible with commodity OS/Apps - require OS/Apps redesign  TCB loses its assurance - code becomes large and complex

Copyright © Step 1: Separate Step 2: Mediate Step 3: Emulate GPU Separation Kernel (GSK)

Copyright ©  Separate security-sensitive from insensitive GPU objects => security model (informal) GSK: Separation App 1 OS (unmodified) Apps GPU

Copyright © 2015 Insensitive (vast majority) 13 GSK: Separation Sensitive Object Insensitive Object App 1 OS (unmodified) Apps  Separate security-sensitive from insensitive GPU objects => security model (informal) GSK Sensitive (very few) GPU Addressed: Large and complex (unverifiable) code base

Copyright ©  ALL accesses to security-sensitive objects by ALL GPU instructions inadequate GPU HW for mediation and complex instruction behavior Interfaces for trusted display GSK: Mediation GPU App 1 OS (unmodified) Apps Access Mediation SecApp 1 GSK

Copyright ©  cannot be intercepted by GPU during execution  can access global memory via global page table (GGTT) can access all frame buffers  have complex behaviors when accessing sensitive objects  Assign GPU instructions to separate address spaces  Prevent GPU instruction access to sensitive objects while maintaining compatibility.  Map GPU instruction behaviors to Read/Write & Config. Change accesses. Enforce access invariants. Inadequate GPU HW & complex behaviors Solutions Instructions GSK: Mediation

Copyright © GPU Address Space Separation GPU Instructions Global Page Table (GGTT) Physical Memory Sensitive Object Insensitive Object

Copyright © GPU Address Space Separation GPU Instructions Global Page Table (GGTT) Physical Memory Sensitive Object Insensitive Object

Copyright © GPU Address Space Separation GPU Instructions Global Page Table (GGTT) Physical Memory Shadow GGTT (GGTT’) Sensitive Object Insensitive Object Addressed: Inadequate GPU HW and access mapping

Copyright ©  Preserves compatibility of access to shared objects e.g., both OS/Apps and GSK access the frame buffer base register GSK: Emulation Interfaces for trusted display GPU App 1 Apps SecApp 1 GSK Access Mediation Emulation OS (unmodified) Addressed: Incompatibility with commodity platforms

Copyright ©  Relies on existing primitives of formally verified μHV - access control to CPU physical memory GSK: Design GPU App 1 OS (unmodified) Apps Access Mediation SecApp 1 Emulation GSK Addressed: Maintain assurance of underlying code micro-Hypervisor

Copyright © GSK: Design OS/Apps frame buffer SecApps’ frame buffer Screen Addressed: Maintain Users’ Perception  Screen Overlay: displays SecApps over OS/Apps

Copyright © 2015 GPU ObjectAll Objects Mediation in Full GPU Virtualization GSK Data (e.g., frame buffer, input/output for processing) 2 GBdata “out-of-the-VM” ~6 MB Configuration Registers Page TableAll Commands Instructions6614 (Ignored)0 22  Only few GPU objects require mediation  Much smaller trusted code size << GSK + μHV << Full GPU Virtualization ~36K SLoC >10M SLoC Evaluation: Size & Complexity

Copyright © μHV-only μHV + trusted display Un-optimized μHV causes most overhead Evaluation: Performance (Throughput)

Copyright © Evaluation: Performance (Latency) Native μHV + trusted display (ms) μHV only (ms) Un-optimized μHV causes most frame jitters (frame)

Copyright © Take-Away Points  Trusted display: Secure Compatible with commodity software/hardware Preserve assurance of underlying trusted code Maintain a typical user's perception  Approach: Separate  Mediate  Emulate GPU accesses Screen overlay

Copyright © Backup

Copyright © Security Protection Sensitive App (SecApp) Operating System (OS) App Keyboard Graphic Controller … Network (w/ crypto) Server ! Sec- App

Copyright © Discussion  SecApps require GPU acceleration Need to extend the scope of sensitive GPU objects Still simpler than full GPU virtualization  GPU hardware enhancement Separate sensitive and insensitive GPU registers and memory into different aligned pages Support R/W access control in all GPU page tables

Copyright © OS/App frame buffer 1 Screen SecApp frame buffer 2 Challenge: Ideal Trusted Display when Screen & GPU are Shared at Any Time (not exclusively) SecApp frame buffer 3 … Screen Sharing

Copyright © Evaluation: Performance (Latency) Native μHV + trusted display (ms) μHV only max acceptable latency (ms) Un-optimized μHV further degrades user experience (frame)