1. Implementing & Troubleshooting PVS
Gareth O’Brien
October 2012

2. Introduction Gareth O’Brien, escalation engineer
Intermediate to advanced PVS knowledge expected

3. Agenda Planning Installing Configuring Troubleshooting
Database, storage, network, farm layout
Installing
Configuring
Files needed for booting, boot process
Troubleshooting
Stream Service
Console, MAPI and IPC

4. Implementing PVS

5. Planning Planning the database configuration Hardware requirements
Client & Server considerations
Network
Farm layout considerations

6. Database Size of farm determines database server choice
Fault tolerance and failover
Mirroring
dbOffline – if all else fails
SQL native client needed for mirroring
Permissions
Dbcreator & securityadmin for Configuration Wizard
Db_datareader, db_datawriter & execute permissions on stored procedures

7. Hardware Requirements
Supported Operating systems
Windows Server 2003 R2, 2008, 2008 R2
Minimum spec of the PVS server
Known hardware issues
NIC Teaming
Make sure hardware is not on above list
If streaming doesn’t work with team, try break the team
Processor: 3GHz
RAM: 4GB
NIC: 1GB per 250 targets

8. Network Configure UDP & TCP Ports
Server to server ports must be the same on all servers
Target device communication ports
SOAP Server
54321 & 54322
TFTP & TSB
69 & 6969
Firewall must be configured to allow all these ports
If in doubt, allow ANY/ANY to test
PortFast must be enabled
Blocked
Listen
Learn
Forward
Disabled
From blocking to listening: 20 seconds
From listening to learning: 15 seconds
From learning to forwarding: 15 seconds
PortFast dramatically reduces this time. needed for DHCP, DNS & PxE (among others)

9. Plan the farm Site 1 Site 2 Provisioning Services Farm License Server
MSSQL Server Database
Shared Storage
Site 1
Device Collection
Provisioning Servers
Provisioning Servers
Device Collection
Site 2

10. Installing PVS

11. Installing PVS Server installation Console installation
If installing a new version of .NET, reboot when prompted
Specify the mirror failover if applicable
Console installation
Needs PowerShell 2.0
If upgrading, make sure you remove the old version
Target installation
Make sure you set the target to boot from network first
If imaging fails
Start with a vanilla install & try to image. Remove any teams
XenConvert & BNImage are other options

12. Configuring PVS

13. Configuring PVS PXE Boot Two Stage Boot (TSB) Boot Process Stores
IPC Key

14. PXE Boot Must configure DHCP Uses TFTP Protocol
Option 66: IP of PVS Server
Option 67: Name of bootstrap (ardbp32.bin)
Uses TFTP Protocol
TFTP listens on one NIC only
Configure using c:\program Files\citrix\provisioning services\tftpcpl.cpl
TFTP Downloads bootstrap, ARDBP32.BIN
Other TFTP servers can be used

15. Two Stage Boot (TSB)
Used in situations where DHCP or PXE cannot be used
Uses ISO burned from PVS Server
Uses a boot strap called tsbbdm.bin
Listens on NIC configured for streaming in configuration wizard
Uses port 6969
Uses Two Stage Boot service on PVS server

16. Boot Process
Target device either uses DHCP to obtain an IP or loads static IP (for TSB)
Once it has an IP, it will contact the 1st logon server specified in the bootstrap
The logon server will then create a context for the device, determine which vDisk the target is assigned and the least loaded server
This calculation is based on both RIMS information and the load information in the database
RIMS uses notifier and inventory service to update load information
If subnet affinity is enabled, this will alter the calculation
Once a server has been selected, a streaming IP and port will be provided
IP information will be in HEX, not decimal
= C0A801
= AC12
10.0. = A0

17. Boot Process (cont) Excerpt from log of a target logging on

18. Boot Process (cont)
As streaming begins, the boot control device (BCD) is downloaded
This phase is referred to as single I/O
Once Single I/O has finished, we enter the admin phase
BNIStack handshake
Vdisk configuration (name, mode, write cache type) is requested and provided
Target device requests personality string
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BNIStack\Parameters
DWORD EnableGetComputerName set to 1
AD information requested
Multi I/O phase entered

19. Stores Configure vDisk stores Centralised Distributed
Centralised vs Distributed stores
Centralised
Works well in HA environment 
Low storage overhead 
High cost 
Potential SAN/Storage issues 
Distributed
Low implementation cost 
High storage overhead 
Configuration 
IPC is the method for communication between servers for stores

20. IPC Key
Defines which NIC to use for IPC communication in a multi NIC environment
HKEY_LOCAL_MACHINE\Software\Citrix\ProvisioningServices\IPC\
Reg_sz called IPv4Address with the IP of the NIC for IPC
Without it, stores, replication, load balancing etc won’t work
Affects stream service
Manager key for MAPI works the same way
HKEY_LOCAL_MACHINE\SOFTWARE\Citrix\ProvisioningServices\Manager
RegSZ called GeneralInetAddr with the IP of the NIC and port
Eg :6909

21. Troubleshooting PVS

22. A more structured method of troubleshooting PVS

23. Troubleshooting PVS Stream service troubleshooting SOAP/Console
The basics
Tracing
Dumps
Log Analysis
Common Issues
SOAP/Console
Unable to validate
Console timeouts

24. Stream Service Basics The devil is in the detail. Bluescreens
Network Topology
Hardware of target, server & database
Storage type & location
Write cache type
Database location, version, HA
MSInfo32
Bluescreens
Common Image 7B
What does it mean
Common causes
7B = Inaccessable Boot Device

25. Stream Service Tracing
Network traces
Wireshark trace on the server
Tcpdump on the target
Make sure you always use –s 0 to not limit capture size
PVS Logs
Set to trace level
C:\programdata\Citrix\Provisioning Services\Logs
Target side logging also
All logs and traces should be capture simultaneously, while the issue is happening.

26. Stream Service Tracing (cont)

27. Stream Service Tracing (cont)

28. Stream Service Tracing (cont)

29. Crash Dumps If possible, take it in private mode
In 2008, it’s possible to redirect to a non system drive
Not possible on 2003
Configure citrix symbols
Live debugging can be done, but difficult without symbols
Use live debugger to help set it up
Check for BNIstack being loaded
!devnode 0 1
On 5.6 check for bnns
!ndiskd.miniports

30. Common issues Imaging Computer Account
Let the wizard do the work for you
Always revert to a clean install as a first step
If the wizard doesn’t work, pre-create the disk and use xenconvert
BNImage can work well if all else fails
Computer Account
Is AD Management configured on the vDisk? Is Active directory correctly configured?
GPO disabling password update, and password age
Reset the password within the PVS console
Check all relevant Microsoft hotfixes are installed – Google!
BNDevice 1015 in the event viewer – unable to negotiate a new password
Trace logs will show you the requests

31. Performance Follow the best practice guide Intermediate buffering
Intermediate buffering
Generally enabled if free space on target device disk is greater than vDisk, otherwise disabled
Toggling the state is always worth trying to see the performance impact
Check interrupt safe mode is not enabled
Check the network with a network monitor

32. SOAP & Console Console timeout Can’t communicate with other servers
The console timeout can be set in the registry
Use DWORD ConnectionTimeout in HKEY_LOCAL_MACHINE\SOFTWARE\Citrix\ProvisioningServices. Default value is 120s
Can’t communicate with other servers
Check IPC and manager keys
HKEY_LOCAL_MACHINE\SOFTWARE\Citrix\ProvisioningServices\IPC\IPv4Address
HKEY_LOCAL_MACHINE\SOFTWARE\Citrix\ProvisioningServices\Manager\GeneralInetAddr
Mentioned above
Load Balancing issues
Check the StreamProcess & Inventory logs to make sure there are no issues around RIMS

33. Questions?
Remove clustering mention