1. VM Basics – some terminology – Guest: a system that is operating in a virtual machine, a.k.a user – Running under/on VM : running a system as a guest of VM – Running second level: running a system as a guest of VM which is itself is a guest of another VM (running VM in a VM) – A virtual machine may have multiple virtual processors (VPs) Note: LPARS have Logical Processors (LPs) - Virtual Machines and LPARs relationship (see fig VM.1) – Sharing is very important and Isolation is very important 

2. VM Basics – A Simple User Directory Entry USER LINUX01 MYPASS 512M 1024M G //user, mem, class MACHINE ESA 2 // architecture IPL 190 PARM AUTOCR //boot device CONSOLE 01F 3270 A // devices SPOOL 00C 2540 READER * SPOOL 00D 2540 PUNCH A SPOOL 00E 1403 A LINK MAINT 190 190 RR // read only disks LINK MAINT 19D 19D RR MDISK 191 3390 012 001 ONEBIT MW // Minidisks MDISK 200 3390 050 100 TWOBIT MR //Note:A real processor can be dedicated to a virtual machine

3. VM Basics - a few CP Commands CP DEFINE //– Adds to the virtual configuration – CP DEFINE STORAGE //memory – CP DEFINE PROC //processor – CP DEFINE {device} {device_specific_attributes}   CP ATTACH … and CP DETACH... – Gives a real device to a virtual machine or removes it   CP LINK – Lets one machine's disk device also belong to another configuration   CP SET – Change various characteristics of virtual machine   Changing the virtual configuration after logon is considered normal 

4. VM Basics – Sharing processors  - Control and Limits – Scheduler selects virtual processors according to apparent CPU need – “Share” setting - prioritizes real CPU consumption - Absolute or relative - Target minimum and maximum values - Maximum values (limit shares) either hard or soft – “Share” for virtual machine is divided among its virtual processors

5. VM Basics- Start Interpretive Execution (SIE) Permits a guest virtual machine instruction stream to be run When CP dispatches a guest using SIE, details about its execution environment are provided to the hardware. SIE has access to: – A control block that describes the virtual processor state (registers,etc.) – The Dynamic Address Translation (DAT) tables for the virtual machine SIE instruction runs the virtual server until an event occurs that requires CP assistance. At that point, the SIE instruction ends and control is returned to CP for processing.

6. VM Basics - Start Interpretive Execution (SIE)  z/VM gets control back from SIE for various reasons: > the VMs time slice has been consumed > the VM wants to perform an operation that the hardware can't virtualize – e.g (SIE instruction under SIE) > Page faults > I/O channel program translation > Privileged instructions (including CP system service calls) > Other, including CP asking to get control for special cases 

7. VM Basics - Multiple Levels of SIE – Both z/VM and LPAR use SIE – z/VM running on LPAR requires 2 levels of SIE No V=F support, and V=R loses I/O Assist Rest of SIE features can be shared without performance loss – z/VM running on z/VM on LPAR requires 3 levels of SIE A layer of SIE now has to be virtualized... Fairly expensive   2nd level (and 3rd level …) Systems – Often used for testing purposes or disaster recovery

8. VM Basics - Anomalies of Time   VM virtualizes various timers or clocks – CPU timer – runs as processor time consumed – Time of day (TOD) clock – Clock comparator   Anomaly – TOD always moves at wall clock speed – Virtual CPU timer “moves” slower as the sharing of the real processor increases – Problem when calculations assume CPU timer is moving at TOD clock speed 

9. VM Basics -Virtual Processor management   Allows guests to expand or contract the number of virtual processors it uses without affecting the overall CPU capacity it is allowed to consume – Guests can dynamically optimize their multiprogramming capacity based on workload demand – Starting and stopping virtual CPUs does not affect the total amount of CPU capacity the guest is authorized to use – Linux CPU hotplug (cpuplugd) daemon starts and stops virtual CPUs based on Linux Load Average value. -   Helps enhance the overall efficiency of a Linux-on-z/VM environment

10. VM Basics -Virtual Memory Configuration – Defined in CP directory entry or via CP command – Can define storage with gaps (useful for testing) – Can attach expanded storage to virtual machine  Control and Limits – Scheduler selects virtual machines according to apparent need and paging capacity – Virtual machines that do not fit criteria are placed in the eligible list – Can reserve an amount of real storage for a guest’s pages – Can lock certain specific guest pages into real storage

11. VM Basics _ Shared Memory Shared Memory see Fig VM.2   Key Points: – Sharing: Read-only Read-write – Uses: Common kernel Shared programs

12. VM Basics _ real memory layout  See fig VM.3

13. VM Basics - Memory Management   VM – Demand paging between central and expanded memory – Block paging with DASD (disk) – Steal from central based on LRU with reference bits – Steal from expanded based on LRU with timestamps – Paging activity is traditionally considered normal   LPAR – Dedicated storage, no paging 

14. VM Basics - DCSS (Discontiguous Saved Segments) – Defines an address range (MB boundary) to the system – A single copy is shared among all guests – Guest "loads" the DCSS (maps DCSS into its address space) - Can be located outside guest's defined storage – DAT lets this work with minimal CP involvement

15. VM Basics - DCSS (Discontiguous Saved Segments) Contains: - Data (e.g. file system control blocks) - Code (e.g. CMS code libraries) or  NSS (Named Saved Systems) - An IPL-able saved segment – Great for CMS or for Linux - 1 shared copy on system for N guests, instead of N copies. - Faster boot   Special Cases: – Writable by guest, or by CP – Can have both exclusive and shared ranges

16. VM Basics - Dynamic memory upgrade z/VM exploits dynamic memory reconfiguration  Users can non-disruptively add memory to a z/VM LPAR Additional memory can come from: a) unused available memory, b) concurrent memory upgrade, or c) an LPAR that can release memory – z/VM can be reconfigured to reduce the need to re-IPL – Memory cannot be non-disruptively removed from z/VM LPAR   z/VM virtualizes this hardware support for guest machines

17. VM Basics - Device Management Concepts   Dedicated or attached – The guest has exclusive use of the entire real device.   Virtualized – Present a slice of a real device to multiple virtual machines – Slice in time or slice in space – e.g. DASD, crypto devices   Simulated – Provide a device to a virtual machine without the help of real hardware – Virtual CTCAs, virtual disks, guest LANs, spool devices   Emulated – Provide a device of one type on top of a different type – FBA emulated on Fiber Channel Protocol over SCSIS

18. VM Basics- Device Management Concepts   Terminology – RDEV is Real Device Can refer to the device address or the control block – VDEV is Virtual Device Can refer to the device address or the control block – UCB is Unit Control Block Used in hardware definitions – note: RDEV=UCB=subchannel=device=adapter   Control and Limits – Real devices can be “throttled” at device level – Channel priority can be set for virtual machine

19. VM Basics - Data-in-Memory   Minidisk Cache –Write-through cache for non-dedicated disks – Cached in central or expanded storage – Psuedo-track cache –Great performance - exploits access registers – Lots of tuning knobs   Virtual Disk in Storage – Like a RAM disk that is pageable – Volatile – Appears like an FBA disk – Can be shared with other virtual machines – Plenty of knobs here tooS

20. VM Basics- Preferred Guest Virtual machines These virtual servers reside in real memory at fixed storage locations called "zones." (V = R, V = F) Instead of using tables to map virtual to real memory addresses, the hardware simply relocates by applying fixed offsets. (Base -Limits) This zone relocation works together with special I/O assists to enable CP to dispatch a preferred guest with minimal overhead.

21. VM Basics- Networking z/VM meets has several networking options. >CP can dedicate network devices to virtual machines. >Dedicated devices can be: -A channel-to-channel adapter (CTCA) -An Open Systems Adapter (OSA) that provides Ethernet connectivity, or a -HiperSockets device, a network adapter that connects one LPAR to another. z/VM also has its own TCP/IP stack, which guests can use as if it were an IP router. Note: Virtual Networking between LPARS can be faster because messages travel at memory speed

22. VM Basics - Hipersockets HiperSockets is an IBM technology -High-speed communications between partitions - Used with a hypervisor. - Provide in-memory TCP/IP connections between LPARs - Applications perceive no functional differences - Require less processing overhead on either side of the connections, improving performance. - Operate at memory speeds, reducing network latency and improving end-user performance - Reduces real network hops to fulfill lrequests. - Provide security benefits, especially on the memory key protected mainframe

23. VM Basics – A Virtual Network - A common network option is the virtual switch. - Each VM is has a simulated IBM Open System Adapter (OSA) adapter - All the OSAs are connected a simulated LAN segment called a guest LAN. - A real OSA that CP manages is also connected. - CP provides packet- or frame-switching functions for the guests, ( just as a real switch would in a real external network). In this way, the guest LAN becomes an extension of a real external LAN segment.

24. VM Basics- Virtual Devices >CP creates a barrier between the guest VMs and the devices to which CP has access. >One of CP's functions is to mediate access to concurrently shared real devices between two or more guest VMs, such as DASD. > A device may be made available for the exclusive use of a single virtual server, such as a tape drive. >When a virtual server makes an I/O request, CP intercepts the request - translates virtual memory addresses to real memory - examines the I/O request so that no harmful device maintenance requests or subsystem functions are performed, except by authorized servers. Once validated, CP starts the I/O operation.

25. VM Basics – Virtual Devices > If the device is shared, CP intercepts all I/O operations and prefixes the I/O request with additional device controls to further limit the request. > For example, I/O requests to mini-disks are constrained by adding DEFINE EXTENT channel command into the channel program. -This forces the device to limit the I/O request to a certain range of cylinders. > In other cases, a virtual server will be given read-only access to a device. -In this case, CP inserts commands into the I/O request that disable all write-type operations.

26. VM Basics – Virtual devices -Mini-disks are the DISK devices used by virtual servers. -Created by dividing real DASD volume into cylinder ranges. -Appear as separate disk volumes to the VM. -A mini-disk may span a whole real disk volume. Sometimes an entire DISK volume must be dedicated to a VM for its exclusive use

27. VM Basics – Virtual Machine / CP Interface Virtual servers make requests to CP in one of two ways: - A issue CP commands from the virtual server console. - Programs running in the VM, (if authorized by the virtual server OS, may communicate with CP using the DIAGNOSE (DIAG) instruction. The parameters passed with the DIAGNOSE instruction provide all of the details CP requires to obtain input and return a response. T

28. VM Basics – CP Interface The CP command set and functions provided by the DIAGNOSE instruction are divided into "privilege classes." The set user commands and functions intended to be used by all virtual servers: -such as the ability to IPL (boot) a guest OS, -to link to mini-disks, -to display and alter the guest's virtual storage, -and to create and delete virtual I/O devices are confined to the single privilege class G. By design, none of the class G commands can affect CP or other virtual servers

29. VM Basics – The DIAG Instruction Use of DIAG instruction: In architecture the DIAG instruction used as interface to Hardware. If VM, DIAGNOSE instruction is an interface between to VM from virtual server. - If outside its privilege class, the CP rejects the command and returns an error condition to the virtual server (usually the OS). - Used as an request for special IO, shared memory etc.

30. VM Basics - Data Integrity CP and guest VM configurations are under the customer's control, -the actual level of system integrity depends on how the z/VM environment is configured and maintained. e.g. If a mini-disk is made write-shared between two or more VMs, -application or guest OS software needs to manage access to that data. i.e. CP gives the virtual servers access to the same minidisk, it doesn't mediate access between them. Application or guest OS can use the facilities inherent in the I/O architecture or in CP to provide full data integrity. This planning required is no different than the planning required when devices are shared among physical servers.

31. VM Basics - Data Integrity Device functions available to VMs to serialize access to real devices is the same as shared devices among real servers Include: -Reserve and Release - Assign and Unassign. Failure to plan for and implement data integrity functions present in applications or in the virtual server OSs may result in data loss on a write-shared mini-disk.