1. Teradyne License Service and Monitoring Bob Van der Kloot Teradyne, Inc 978-370-2014 bob.vanderkloot@teradyne.com

2. License Service Goals High Availability Minimize effort for maintenance/support –Standardization of operation –Common code across platforms Compliance with license agreements Provide information necessary to manage license portfolio –Easy access to metrics –Standard access

3. License Server Technologies Acresso (Macrovision) FlexLM –The vast majority of our products are licensed with FlexLM. –We support on Solaris and Windows servers –We support complete logging (1 sec resolution) –No LMAdmin!!! Reprise License Manager (RLM) –A new product by the creators of FlexLM, and very similar to FlexLM –We support on Solaris servers –Currently only used by RTDA –We support only daily peak logging, though full logging should be trivial LUM –Now owned by IBM –We support on Solaris servers –Currently only used by SmarTeam –We support only daily peak logging, more may be available via FNM Vendor Proprietary Licenses –MicroFocus/Borland SilkTest License –Rational Atria License (deprecated, but still in use) –Compuware DLM license (not currently in production) –We support only daily peak logging

4. Support Technologies Veritas Cluster Server –Windows and Solaris Veritas Volume Replicator –Windows and Solaris Acresso Flex Net Manager –Linux Master –Windows and Solaris Clients –Oracle Database (Linux Server)

5. Types of License Servers Agoura Hills, California –“means” Solaris 8 cluster Global license service –“cadet” Solaris 8 cluster North America license service –“devways” Solaris 8 cluster Disaster recovery / development Clone of “ways” North Reading, MA –“ways” Solaris 8 cluster Global license service –“adriatic” Solaris 8 cluster North America license service –“wpcluster” Windows Server 2009 cluster Global license service Solaris vs Windows Architecture –Driven by availability of license daemons Global vs North America Service –Driven by Contractual agreements All are Veritas VCS Clusters Solaris vs Windows Architecture –Driven by availability of license daemons Global vs North America Service –Driven by Contractual agreements All are Veritas VCS Clusters Achieve High Availability by: –High availability cluster –Distribute pool between locations when possible 50/50 distribution for vendors that support multiple lic servers Achieve High Availability by: –High availability cluster –Distribute pool between locations when possible 50/50 distribution for vendors that support multiple lic servers

6. License Server Principles Management of licenses is as regular as possible across: –License Server Platform Solaris Windows Linux (when we choose to implement it) –License Technology FlexLM LUM etc… –Cluster or non-cluster All production licenses are on clusters Consistent Abstraction layer provided for automation… same for all licenses –Cluster control Start Stop Health Check Clean –Monitoring List of licenses in consistent format –Quantity –Expiration –Characteristics (UHD, etc) Current Status –Current checkouts –Duration of each checkout

7. License Cluster Basics – Veritas Cluster Server Keep it Simple –Two cluster members Active member: –Hostid is faked –Licenses are served Passive member: –Native hostid of the server –Licenses are disabled –Veritas Volume Replicator for volume containing licenses /data/vol/licenses/ /keys /bin /logs Exact same path on windows server (via links in cygwin) –Standardized control of licenses: Cluster needs to do: –Start –Stop –Health Check User needs to do: –Request cluster to start license –Request cluster to stop license

8. License Daemon Control lmgrdFOO Customization of control of FlexLM daemons Custom each Vendor Administrator (human) lmgrd_generic Standardized control of FlexLM daemons cluster_control Encapsulate Control And Status Check Of Cluster Environment hares VCS cluster software control interface lmgrdFOO Customization of control of FlexLM daemons Custom each Vendor lmgrd_generic Standardized control of FlexLM daemons HA software event detection or Administrator lmgrd Acresso FlexLM daemon control Vendor Daemon Supplies Licenses Custom each Vendor The VCS cluster software uses the same control programs that the human administrator uses (but with different arguments) lmutil Acresso FlexLM daemon control

9. Typical customized calling module #!/bin/sh # This is lmgrdCADENCE # control script for Cadence daemons # this script must be run by "cds“ VENDOR='cadence' /etc/init.d/lmgrd_generic -o $1 -v $VENDOR

10. Command interface of lmgrd_generic -------------------------------- lmgrd_generic version: 1.60 -------------------------------- Usage: lmgrd_generic -o operation -v vendor # vendor name (directory in /data_vol/licenses ) [-k key_spec ] # license file name (.lic implies all.lic files in directory) # (default to license.dat) [-l lmgrd_name ] # lmgrd name override (default to lmgrd ) [-d daemon] # vendor daemon executable name (can replace one only) [-e effective ] # effective daemon name (when daemon runs) [-h hostid] # hostid override [-s sequence] # daemon start sequence ( 0 to 99 allowed ) [-p prefix] # prefix of vendor for control script name [-n] # dongle ( license file will be license_.dat ) [-g] # enable logging [-rev] # print revision and exit [-rlm] # reprise license server (not macrovision) [-w] # windows operation is one of: check - examine files for correct settings start - start license daemon stop - stop license daemon restart - restart daemons (use when license file is changed) clean - clean up from crashed daemon info - report PID numbers of running processes for use only by VCS cluster control: vcs_start - USED BY CLUSTER CONTROL vcs_stop - USED BY CLUSTER CONTROL vcs_clean - USED BY CLUSTER CONTROL vcs_monitor - USED BY CLUSTER CONTROL Written in perl, same executable runs on solaris, windows (cygwin) and linux

11. License Monitoring Goals Standard Interface across all products Maintainable Standard WEB interface useful for beginning and advanced users.

12. Two license monitoring methods used Sample use every 15 minutes –Program “license_use” run remotely on each license server Returns record for each license in use, across all used license technologies vendor|product|version|qty|user|host|display|duration –Data stored in Oracle, plots produced on demand FlexNetManager –Records to-the-second license use, but only for FlexLM Possibly also for LUM, have not tried –Output in “raw” format done once per day, read into Oracle –Plots produced on demand.

13. Demo of license monitor I can’t include slides without violation of vendor license agreement (because it would have real usage data) I’ll show a live demo

14. LSF Batch Environment 6 sites 200+ machines Monitoring by homemade programs –Read LSF log into Oracle

15. What’s Next? Major effort to continue automation and standardization –Reduce Tool and License Administration from 2.5 FTE to ~.6 FTE Significant improvement in prediction tools –Give better access to historic data, so project teams can forecast their future demand.