1. 1 Instant Data Warehouse Utilities Extended (Again!!) 14/7/2004 -- Today I am pleased to announce the publishing of some fantastic new functionality for the Instant Data Warehouse Utilities. These new features have been driven by customer need and discussions on the new yahoo group. They are already in use and I'm sure lots more people are going to download and use these utilities. I hope you enjoy reading about them and choose to use these new utilities. The Batch Processing Scheduling Utility. A number of my customers asked me to produce a utility that would allow them to more effectively manage the batch processing of the Instant Data Warehouse. They asked for something 'better than using a set of scripts'. I thought this was an interesting request and I decided to 'go one step further'. The Batch Processing Scheduling Utility is an extremely flexible utility that allows the user to schedule ANY COMMAND on windows/unix. This means not only can you schedule and run the Instant Data Warehouse and associated utilities. It is now possible to schedule any commands that you want to schedule into any batch on any windows2000/unix machine. In fact, you may choose to use the Batch Processing Scheduling Utility to schedule your other applications!!! It's up to you. The main features of the Batch Processing Scheduling Utility are:  The ability to schedule batches based on a file exists, daily, weekly, monthly and one-time pre- requisites.  The ability to define many pre-requisites for a single batch. For example, both time of day and a file existing.  The ability to break batches up into 'Process Groups' which are then broken up into 'Processes' which are the individual commands to be executed in the batch.  The ability to define file exists pre-requisites for Process Groups.  The ability to run 'Process Groups' in parallel. This is extremely important as the Batch Processing Scheduling Utility gives the user better parallel processing capability than the leading ETL products!!!  The ability to make Process Groups dependent on each other, further enhancing the users ability to construct highly parallelised batches and Process Groups based on the users knowledge of the workload… Press Release From the desk of Peter Nolan

2. 2 Instant Data Warehouse Utilities Extended (Again!!) The DataStage Job Submission Utility. This is also very exciting. I'm sure the DataStage development community is going to be interested in this one!!! As an experienced DataStage developer and ex MVS JCL expert I have always been frustrated at the lack of control that I have over the 'restart after failure' of DataStage jobs. The very idea that a support person has to touch a job or touch DataStage job control language when a failure happens has always filled me with fear as to what mess might be created by making a mistake making these changes. Those of us who came from the large systems environment of complex batch processing knows that the only thing you want to tell a system that has crashed is 'Restart'. The system itself should be able to figure out where and how to restart. This has never been the case with the leading ETL tools......until now!!!! The DataStage Job Submission Utility is a simple command line application that can submit a job on ANY DATASTAGE SERVER that can be seen across the network. However, when used in conjunction with the Batch Processing Scheduling Utility it is now possible to gain far greater control of the parallelisation of jobs running on one or more DataStage servers. Press Release From the desk of Peter Nolan  The ability to restart failed batches by simply telling the Batch Processing Scheduling Utility to restart the batch. The Batch Processing Scheduling Utility will determine what Process Groups must be re-run and inside the Process Groups the Batch Processing Scheduling Utility will determine what processes need to be re-run. The 'abended' process is restarted from the beginning.  The ability to write the start/top timestamps for Batches, Process Groups and Processes to database tables as logs so that the user can see the elapsed time of Batches, Process Groups and Processes and tune the batches to make the optimal use of the available resources by minimising wait time between dependent processes.  Simply put, the Batch Processing Scheduling Utility is a full function scheduler for command line applications. Given that no such thing exists in Windows 2000 I know a lot of people who would like to try this utility out!!!! Watch this space, there is a GUI coming for the Batch Processing Scheduling Utility. The GUI will make it even easier to construct Batches.

3. 3 Instant Data Warehouse Utilities Extended (Again!!) With these new utilities the User Guide for the Instant Data Warehouse has now passed 150 pages and continues to grow. You can read the complete documentation of these utilities in the new 150+ page user guide by going to my Downloads Page. Best Regards Peter Nolan Press Release From the desk of Peter Nolan Further, with the ability of the Batch Processing Scheduling Utility to run the DataStage Job Submission Utility it is now possible to (almost) eliminate the need for DataStage job scheduling code!!! And the Batch Processing Scheduling Utility can automatically determine which jobs to restart in the event of failure. From today, there is no longer any need to make any changes to any DataStage code in the event of needing to restart from a failure. (Apart from fixing the offending job, of course!!!) I am personally convinced these two utilities when used together will make life much easier for DataStage developers. I hope that the DataStage developer community will use and enjoy!!!!