1. Overview of IBM Spectrum Protect for Virtual Environments (TSM4VE)

2. Agenda
Overview of IBM Spectrum Protect for Virtual Environments (TSM4VE)
Product Components Overview
Tivoli Storage Manager Server Backend Structure
Backup and restore types
Backup Transport Options
DataMover (aka vStorage Backup Server)
Options File (dsm.opt)
Client Schedules
Log Files, Stats and Reporting
Application Aware-Backups
Issues and Common Error Codes
Extras
Questions

3. Steven Soave Twitter: @StevenSoave
Who am I ?
Steven Soave
Current Position: Wintel & VMware Technical Specialist
Worked in IT for around 14 years (VMware products around 8 years)
VMware Certification: VCP6-NV

4. Disclaimer
The information in this presentation are from my own experiences and do not reflect the views or configures of the product at my employer.

5. Poll
Who has used/is using IBM Spectrum Protect for Virtual Environments?

6. Overview of IBM Spectrum Protect for Virtual Environments (TSM4VE)
What is it?
Image level virtual machine backup product leveraging VMware snapshots/change block tracking (CBT)
Who are some of its direct competitors?
Veeam
CommVault
Veritas NetBackup
Supports VMware (vCenter / vCloud Director) and Hyper-V - (Windows and Linux VMs)
Working with this product since 2014 when it was called “Tivoli Storage Manager for Virtual Environments 6.4 (TSM4VE)” to present where it is now called “IBM Spectrum Protect for Virtual Environments 8.1” (name changed circa v7.1.3)
Licensing model I’ve been exposed to: ‘data volume licensing’ – which means you can use all the IBM Spectrum Protect suite of products. Hence in essence corporations that have this licensing won’t let you use anything else as they will see it as paying double (through experience) 
(cont.)

7. Pre using an imaged based backup solution like TSM4VE all my experiences were backing up at the file level.
With TSM4VE if all the stars are in aligned its been known to achieve 100% backup success rates….. For a day or two ;-)
More on this later as “It Depends” on a number of components all working optimally

8. Product Components Overview
Physical Components Overview

10. Configuration View
DataMover Services View

11. Some Firewall Stuff (FYI)

12. Tivoli Storage Manager Server Backend Structure
Policy Domain (Web Console Links to the Default Management Class)
When using the web console you can only manage the default management class, schedules, backups and restores etc.
All nodes (backup endpoints i.e. TSM agents on servers or DataMovers) belong to the Policy Domain
If you want to use an alternative Management Class it needs to by specified in opt file and use CLI to perform backups and restores
Backup Retentions - 35 days
Storage Pools - Disk based, not tape (you can use tape but I highly recommend you don’t)
Storage Pools are linked to the management class

13. Backup and restore types
Periodic Full
Full backup
Useful for long term backups
Incremental forever
An incremental forever backup strategy minimizes backup windows while providing faster recovery of your data.
Performs an automatic Full if one doesn’t exist
Reduces the amount of data that goes across the network.
Reduces data growth because all incremental backups contain only the blocks that changed since the previous backup.
Reduces the length of the backup window.

14. Full VM Restore
Restore a full or incremental VM backup. The entire VM is restored to the state it existed in when originally backed up.
Note: When restoring a VM the original vNIC MAC addresses are retained, as long as the original VM has been deleted or the vNIC removed from it so the system doesn’t detect a MAC conflict.
Full VM Instant Restore
The restored VM becomes available for instant use, either for validating the backed up VM or for restoring the VM to permanent.
Basically gives you instant access while the full restore is happening in the background.
Full VM Instant Access
The restored VM becomes available for instant use for validating the backed up VM as at the restore point specified.
Any changes while using this restore will be cleaned up when the restore point is unmounted, no changes are written back to the backup.

15. VM Restore GUI
Limitation: No search option (need to find where the VM currently is using vCenter)
CLI: If restoring using CLI commands then you just specify the VM name

16. File Restore Methods
The are three methods to do file level restores. All three use the “Recovery Agent” and iSCSI
Method One: Using the “Mount Proxy” via the Web Console

17. Method Two: Using the Recovery Agent directly
Individually select and mount volumes required
No indicator as you what the OS logical volume label is
Is mounted as a logical disk/disks on the DataMover OS to browse the backed up files

18. Method Three: Using the File Restore UI web console (End User Self Service)

19. Backup Transport Options
dsm.opt = vmvstortransport option to default (san:hotadd:nbdssl:nbd)
san
You want to offload backup traffic from the LAN, and instead, move it over a SAN.
Required the DataMover to be a physical server
hotadd
You are using a data mover node that is installed on a virtual machine to back up other virtual machines.
Mounts target VMs vmdk using SCSI on the DataMover.
Maximum of 15 SCSI disks per controller and a maximum of four controllers = 60 SCSI disks.

20. ndb
You are backing up virtual machines over an Ethernet LAN; you either do not have, or do not want to use, a SAN to offload backup traffic from the LAN.
Uses port TCP 902 Network File Copy (NFC) from the ESXi host management interface to the DataMover. Note: if using any bandwidth restricting technology (BladeSystems) then ensure you have taken this into account as it could elongate the backups.
nbdssl
You are backing up virtual machines over an Ethernet LAN and you want to use SSL to encrypt the data..
Note that encrypting data can degrade backup performance

21. DataMover (aka vStorage Backup Server)
Proxy Sizing
General rule of thumb is to use a data mover for every 100 TB of vSphere data
10 GbE (or HotAdd / SAN equivalent) available on all data paths in the environment, specifically the path from the datastore to the data mover and then to the IBM Spectrum Protect server
5% average daily change rate
8 hour backup window
Data mover hardware specification The following hardware specification is suggested for each data mover:
16 cores (2.8 GHz) if using client-side deduplication
8 cores (2.8 GHz) if using server-side deduplication
8 GB RAM
10 GbE network

22. DataMover Nodes are service instances configured via the TSM client agent on a VM or physical server:
TSM Client Acceptor Service (dsmcad.exe) = Handles comms with TSM server and listens for connection from Remote Client Agent to start a backup or restore. Also listens for prompted schedule changes on behalf of the Client Scheduler.
TSM Client Scheduler Service (dsmcsvc.exe) = Handles the Client Schedules created on the TSM server and running them
TSM Remote Client Agent Service (dsmagent.exe) = Used to run Run Once and Scheduled jobs when prompted by the Acceptor service
Multiple DataMover instances can run on a single server with varied dsm.opt instruction sets which can by linked to certain client schedules.
Not a good idea to run multiple DataMover servers that target the same client schedule at the same time in order to decrease backup time.
DataMovers can target vCenter Clusters, Hosts, VMs, Datastores or Folders

23. ‘dsm.opt’ file options
VMBACKUPTYPE FULLVM VMCHOST <vCenter server name> VMCUSER <vCenter user account> SCHEDMODE PROMPTED ERRORLOGRETENTION 35 D SCHEDLOGRETENTION 35 D COMPRESSION YES COMPRESSALWAYS YES DEDUPLICATION YES *DEDUPCACHESIZE 2048 *DEDUPCACHEPATH "C:\Program Files\Tivoli\TSM\baclient\TSMDEDUPDB\STD" ENABLEDEDUPCACHE NO VMENABLETEMPLATEBACKUPS YES VMPROCESSVMWITHPRDM YES VMPROCESSVMWITHINDEPENDENT YES VMSKIPMAXVIRTUALDISKS YES VMMAXVIRTUALDISKS 8 VMLIMITPERDatastore 2 VMLIMITPERHost 8 VMMAXParallel 12 VMVSTORTRANSPORT "NBD:HOTADD" TESTFLAG VMBACKUP_UPDATE_UUID VMVERIFYIFAction FORCEfull INCLUDE.VMSNAPSHOTATTEMPTS * 1 1

24. VMCUSER Permissions in vCenter (Create a Role) http://www-01. ibm

25. Client Schedules Unsupported characters in VM or datacenter name
Data Protection for VMware does not support backing up VMs or datacenters that contain any of the following characters in their name (ignore the brackets)
(“) Double quotation mark (‘) Single quotation mark (:) Colon (;) Semicolon (*) Asterisk (?) Question mark (,) Comma (<) Less than sign (>) Greater than sign
(/) Forward slash (\) Backward slash (|) Vertical bar
Schedules are associated with a DataMover instance (service)
No other schedule can be run using a DataMover instance that currently has a scheduled job running. Hence you will see the need for multiple DataMover instances if breaking out longer running backups etc.
DataMovers can have multiple schedules assigned to it. (Note: Care must be taken as the later schedule may not run if current schedule run time supersedes the latter's start time (see Schedule Duration)
Schedules have Duration Windows. If schedule is due to run but another is still running then the latter will still run as long as its start time is within: - Start time + Duration Window
Two types of schedule: - Run Now
- Scheduled

27. Log Files, Stats and Reporting
DataMover Log Files are you friend and I use them daily for troubleshooting:
Located here by default: C:\Program Fiels\Tivoli\TSM\baclient
Most used logs (
dsierror.log = Used to show tsm server errors
ANS0266I The dsmEndTxn vote is ABORT, so check the reason field.
ANS1311E Server out of data storage space
ANS1301E This operation cannot continue due to an error on the TSM server
dsmerror.log = Used to log errors by each of the client applications: dsmc, dsmadmc, dsmcad, dsmj, Web Client
ANS0361I DIAG: VmVerifyAllBitmapCtlAreInJobGroup(): Ctl file from bitmap.dat is missing in job folder! MegaBlock number:11299 Job number:550005
ANS9919E Failed to find the expected control files for TESTVM
ANS9920W Forcing a full vm backup for TESTVM
ANS9386W Changed block tracking is not supported for virtual machine TESTVM2'. A FULL backup of the disk will be performed and will include both used and unused areas of the disk.
dsmsched.log = Used to write client schedule information, and scheduler errors for applications: dsmc scheduler, dsmcad
:27:16 Processing snapshot
disk: [DATASTORE_NAME] TESTVM/TESTVM_1.vmdk (Hard Disk 2)
Capacity: ,949,672,960
Data to Send: 3,023,110,144
Transport: (nbd)[sending]
dsmwebcl.log - Used to write Web Client messages and Client GUI messages for applications: dsmcad, dsmj, Web Client.

28. Typical Schedule Run Stats for a DataMover targeting one Production vCenter cluster
:30:51 Accessing as node: DATACENTER NODE
:30:51 Aggregate statistics for Backup VM command.
:30:51 Total number of objects inspected:
:30:51 Total number of objects backed up:
:30:51 Total number of objects updated:
:30:51 Total number of objects rebound:
:30:51 Total number of objects deleted:
:30:51 Total number of objects expired:
:30:51 Total number of objects failed:
:30:51 Total number of objects skipped:
:30:51 Total number of objects encrypted:
:30:51 Total objects deduplicated:
:30:51 Total number of objects grew:
:30:51 Total number of retries:
:30:51 Total number of bytes inspected: TB
:30:51 Total number of bytes processed: TB
:30:51 Total bytes before deduplication: TB
:30:51 Total bytes after deduplication: GB
:30:51 Total number of bytes transferred: GB
:30:51 Data transfer time: , sec
:30:51 Network data transfer rate: , KB/sec
:30:51 Aggregate data transfer rate: , KB/sec
:30:51 Objects compressed by: %
:30:51 Deduplication reduction: %
:30:51 Total data reduction ratio: %
:30:51 Elapsed processing time: :55:16
:30:52 Backup VM command complete
Typically backup (incremental) approx VMs over a 10 hour backup window, multiple TSM4VE environments

29. Reporting: Predatar (SilverString)
TSM Manager

30. IBM Spectrum Protect for Virtual Environment Web UI

31. Application-Aware Backups
Supports
Microsoft Exchange Server
Microsoft SQL Servers
Microsoft Active Directory Domain Controllers
More info:

32. Issues and Common Error Codes
Consolidating VM backups
After a VM backup, the VM might contain pre-existing snapshots even though no snapshots are present in the Snapshot Manager. For example, the VM hard disk points to snapshot VMDK files (for example * vmdk) instead of regular VMDK files. 
Rare: When removing snapshot after backup VM is powered down
When logged with VMware they say its Storage I/O / timeout related where vCenter will power off the VM by design?
Be aware of snapshot stun times, I’ve known this to affect SQL servers mostly. I believe this is better in vSphere 6 >
Centrally (Help Desk) managed file restores is a major pain to achieve. Lots of support calls with IBM support
RC TCP/IP disconnect issues between DataMover and TSM server instance
RC Kind of a false positive. Quiesce snapshot failed but crash consistent successful (net result VM backed up)
RC Target specific VM in schedule and VM no longer exists in vCenter inventory
RC 11 - Normally Storage Pool is full
RC Normally issues around VMTools

33. Extras
New version of this product called “IBM Spectrum Protect Plus”
Includes RESTful API
New file restore (search and recovery)
Rumoured Q1 2018
IBM Spectrum Protect for Virtual Environments as of v8.1 can now do:
Enhancements for backup and restore performance for VMs: Optimized backup operations for multiple virtual disks by using parallel sessions, one session for each virtual disk Optimized backup operations for a single virtual disk by using parallel sessions, multiple sessions for each virtual disk
VVols Support
Veeam Availability Suite – Not-For-resale (NFR) License
If you have a Home Lab and need a backup solution check this out!

34. Thank you Questions