1. Managing Systems With Ignite-UX
Bob Campbell
Technical Contributor
Patch Clearinghouse (PCH)
Hewlett-Packard

2. When to Ignite Similarity is a good thing Advantages in numbers
cookie cutter clones
performing same functions
matching architectures
Advantages in numbers
minutes saved add up quickly
less desire to customize
infrastructure cost less of an issue
If all systems are identical in every way, they are perfect candidates for managing by IUX. Few of us are in that position, but IUX golden images can support differences in hardware to a fairly large extent. It is still best to group systems by common purpose, such as all web servers. The ultimate requirement is that the processor architecture matches. This means 32- and 64-bit systems need different images.
Time saved directly by igniting is often measured in minutes. For a single system the setup may not be worthwhile, but if 50 systems are managed the total savings could well be measured in days. If you are managing 50 systems you are also less likely to attempt to manage them one-by-one, and a dedicated system for use as an IUX server is not likely to blow the budget.

3. When to Ignite? Critical Time Limits
limited maintenance windows
predictable install times
automation friendly
PCH full ignite in 12 minutes
rapid fallback required
IUX server can host old images
known ignite time for previous image
full installation allows alternate disks to be used
An ignite archive is basically a large compressed tar ball. It is about as fast as a serial installation can be. Unlike SD, there are very few scripts that need to be executed as a part of the installation process and the execution time is very repeatable.
IUX was built to allow HP manufacturing to preinstall systems prior to shipping. This leads to it supporting an automated process well. Almost all of the PCH usage is controlled from our automation.
While a cold-install from SD depots would take minutes and place a relatively high load on our depot server, IUX can install the same system in less than 15 minutes and while loading the network and disks at a higher level, they run for less time with less CPU.
A maintenance window has two ends. The ability for the IUX server to host many images at once with the reproducibility of installation times allows a fail-safe point to be put into maintenance windows that restores the system to full function in the old environment. The best way to do this is to use the full system image load to its full advantage and alternate image loads between two different boot disks. This leaves the previous environment a single reboot away.

4. When to Ignite? Wiping the slate clean
restoring test systems
eliminate cross-contamination
better test reproducibility
systems of many hats
ignite to new function
full environment switch
share hardware, not information
share a useful system, without data
If you have test systems, modifications made during an earlier test can interfere with subsequent tests. By igniting test systems prior to each run, the PCH assures that we start from a known, good state and can expect to be able to reproduce our results in later runs.
Our test systems also support 10.20, 11.00, and releases. When a patch comes in a test system is ignited to the proper environment. Each system does not have to be sized in order to host all tests at once.
The obvious example of the last item is leased systems. Between customers, an ignite assures no data or security holes are left behind while keeping the system in a usable state.

5. Configuring for Success Keep your roots clean
no volatile data on root volume
avoid loss or backdate
takes extra time to restore
user directories automounted
biggest source of root data
NIS for configurable data
avoid need for customizations
Igniting a system is a full reload of the root volume group. If data that is not part of the system software is kept in the root volume it will have several impacts. First, when the image is loaded this data can be lost or backdated. Restoring from other backup media or including the extra data in the archive load costs time.
For many of my systems, the user files under /home is a source of root data. While not a problem for many production boxes, we eliminate the issue by hosting all user directories from a central NFS server and move volatile data such as password files off-system using NIS.

6. Configuring for Success Don’t skimp on the server
fast disks; multiple io cards
image files are gigabytes in size
disk spindle is #1 bottleneck
fast networking interfaces for all subnets
server should be faster than any client
gateways add to impact and complexity
double duty as net_recovery host
justify as a disaster recovery expense
Disks are slow, but some are slower than others. The disk space and configuration of the devices and file systems should be such that bottlenecks are avoided. Remember that even when compressed these archives are large files and there are benefits in keeping old ones around.
An IUX server and client can fill their network capacity, and it is often useful to have several clients igniting at once. If you need to pick one place to put that gigabit card, and IUX server could put it to good use.
Remember that the IUX server can be used even with systems that are not ignited. The IUX net_recovery tool is a wonderful thing when software or hardware failures bring a system down. Rather than diagnose what went wrong on a production server, use a recovery image to get it back to where it was.

7. Configuring for Success Made for each other
Build your systems for images
HW arrays are faster than SW mirrors
allow for two sets of root volumes
remote console access
Build your images for systems
include needed drivers and HWE patches
beware cross-volume symbolic links
block bootsys automatically
IUX does not support software mirroring, but can ignite to an array. If software mirroring is used, then time will need to be allocated to allow the mirror volume to be brought online.
If two bootable volumes can be maintained, recovery by rolling back is only a single reboot away.
A system can be ignited directly from the console even if the existing disks are not bootable. While the PCH does most of our work remotely, the amount of ignites we do has shown that being able to quickly grab consoles allows us to address problems quickly.
Be sure to put full support for systems and cards to be used on the image. Keep in mind new purchase that might appear during the life of the image. Beware cross-volume symbolic links that leave files off of the root volume. We try to always build our images from a system that has been freshly installed.
Our automation runs ignite for us. If you are working with a similar environment and production systems, it is best to block subsequent ignite sessions during the ignite itself. This is done by creating the file /.bootsys_block.

8. Then the Real World Hits…
We all know that:
almost cookie cutter doesn’t count
getting one user to decide is hard enough…
tinkering happens
You will be customizing

9. Golden Image Configuration
IUX configuration files
For defining the root volume, setting up networking, and importing data volumes
User-defined scripts
Probably already exist, written in common language (sh, perl…), modified outside IUX
User-defined depots
SD depots with software to load on image

10. Release-specific config file
located in /opt/ignite/data/Rel_release
set the defaults for HP-UX release
great examples for beginners
The language used by IUX config files is defined in the instl_adm(4) page

11. Server-specific config file
located at /var/opt/ignite/config.local
provides info for use from specific-IUX server
dns_domain="fc.hp.com"
dns_nameserver[0] = " "
dns_nameserver[1] = " "
nis_domain="corp"
wait_for_nis_server=FALSE
ntpdate_server=" "
The IUX server lives in a specific location on specific network(s). The server-specific configuration files are used to set up values that apply to all systems installed using that server. The obvious example is networking.
In the excerpt shown, the DNS domain, name servers, NIS domain, and time server are all configured.

12. System-specific config file
/var/opt/ignite/clients/0xLLAddr/config
can be used to set primary & secondary swap, path to boot disk, keyboard type
init _hp_disk_layout="HP-UX save_config layout"
init _hp_root_disk="8/4.5.0"
init _hp_min_swap= K
init _hp_locale="SET_NULL_LOCALE"
init _hp_keyboard="PS2_DIN_US_English"
not only method for system-specific changes!
( system_name == "chc36001" ) { }
Each system may define its own config file. Kept distinct using the link-level address of the networking card used to ignite the client, it is the final config file to be loaded and as such gets the last word.
In this example, we specify the root disk, minimum swap required, the locale, and type of keyboard. This file is not the only method that can be used to specify system-specific items. Other config scripts can check for the system being ignited and take appropriate action using a simple test as shown here.

13. post-installation scripts
set using post_config_script variable in config file, or defined in INDEX file using the scripts clause
selected scripts will be run on target system during final configuration
scripts can be run prior to the initial reboot using the post_load_script variable, but limited commands are available for use

14. Using IUX from scripts
bootsys utility allows remote systems to be prepared and rebooted to IUX control
script is outside of IUX, and can be built with knowledge of multiple servers
remote console access is strongly recommended to allow quick recovery from minor problems

15. Using Delta Depots image maintenance
minor corrections without rolling image
security patches, patch warnings, critical or locally required newer patches
shared image for similar environments
optional core components & applications
patch level management
paid for in installation time, use wisely

16. /var/opt/ignite/INDEX
A cfg clause in the INDEX file will appear as one selection in the IUX GUI. This defines the 32-bit image used for workstation ignites in the PCH:
cfg "CH_11.11_32_700_HF" {
description “PCH 11.11_32_700 archive"
"/opt/ignite/data/Rel_B.11.11/config"
"/var/opt/ignite/config.disks_700"
"/var/opt/ignite/images/1111_32_700.cfg"
"/var/opt/ignite/depots/latest_patch.cfg"
"/var/opt/ignite/config.local" }
The release-specific configuration is specified first. The values that are defined may be overwritten by changes made in the later files.
Next is the config file that defines the file-system layouts for our test systems. While these values could have also been set in system-specific files, we chose to define our disk layouts in a common file for all workstations and another for all servers. Selections are made based on disk and memory size and in some cases system ID.
The golden image to be used is defined in its own config file (1111_32_700.cfg) that specifies how the image is found and transferred to the client. It also records the space requirements and layouts for the default file systems. This file is generated by make_sys_image.
At times it may be convenient to update or modify an image indirectly by installing additional software from an SD depot. In this case, additional patches needed for our testing are found in a dedicated depot. The config file (latest_patch.cfg) was built using the IUX make_config utility.
The last file in the list is the IUX server-specific config file. While it is not in the list, the client-specific file always will be loaded last and will have the final say.

17. IUX recovery images
archive
archive, control, and system config files
A recovery image consists of the archive of all files, and three configuration files that define it and the system that created the image

18. IUX recovery images
The archive can be prepared for installation on another system by creating new config files based on the new target (a copy of the archive is not needed)

19. The maintenance system
After loading image on the maintenance server, any updates desired are installed and tested before a new image is created for the updated environment

20. Once more, with feeling!
The new archive uses the same process to restore the configuration files for installation on the original production box

21. The ideal ending
By loading the updated image without destroying the original boot volume, recovery is a reboot away. The data disks are rehosted in either environment thanks to the config files.

22. IUX Recovery for IPF Systems
No Tape Boot Support in IPF Firmware
No EFI tape device drivers
No sequential tape boot control firmware
Future Media Recovery may be Disk
Increasing capacity of DVD media and widespread use may lead to use as the future recovery media

23. IUX Recovery for IPF Systems
IPF System Two-Step Media Recovery
Step 1: Boot from CD
Step 2: Recover from tape
Use Cold Install CD for selected OS
Step 1 CD is any HP-UX install CD for the OS version of the recovery image
Support Starting in HP-UX 11.23