1. 1 Instant Data Warehouse Utilities Extended 2/4/2004 -- Today I am pleased to announce the publishing of some promised new functionality for the Instant Data Warehouse Utilities. These changes have been mostly driven by requests from customers and discussions on the new yahoo group. In summary the changes are:  A new utility called the Generate Delta File Utility. This utility makes it possible to compare an 'old' and a 'new' file and generate the transactions that must have happened to the 'old' file in order to produce the 'new' file. In short this utility is a 'delta' generation utility. So, for source systems that cannot provide change data capture this utility allows the user to implement the ability to generate 'change data capture' at the file level. This is a very important utility for customers who are not able to detect changes to some tables in production systems. The Generate Delta File Utility can work with the Delimiter Separated Values Reformatting Utility and the Fixed File Format Reformatting Utility (see below) to enable the user to perform delta processing on files extracted from source systems without actually having to load the data into a database in order to compare the 'old' and 'new' versions of the data. A major reduction in processing requirements over performing the comparison using a database to support the comparisons. Press Release From the desk of Peter Nolan  A new utility called the Delimiter Separated Values Reformatting Utility. This utility makes it possible to load Delimiter Separated Values files into a staging area to be processed by the Instant Data Warehouse into the Data Warehouse. The most common DSV being the standard CSV files produced by applications like Excel.  A new utility called the Fixed File Format Reformatting Utility. This utility makes it possible to load fixed format files, commonly extracted from mainframe systems, into a staging area to be processed by Instant Data Warehouse into the Data Warehouse.  The Data Transfer Utility has been upgraded to be able to produce a 'Load Interface File' which can be used by standard native database loaders to speed up the loading process. This has been particularly added for Sybase IQ. So those of you using Sybase IQ 12.4.x can now load the database from a delimited file…

2. 2 Instant Data Warehouse Utilities Extended  There has been a general upgrade of all programs to enable:  Handling of translation of newlines so that they can be read into the Instant Data Warehouse and loaded into the data warehouse even though the internal self describing file format uses newlines as the line delimiter.  The ability to set the delimiter to be used inside the self describing file format as a parameter. (Yes, someone wanted to use '~' in their data!!!)  The ability to translate a character from one character to another. This is to translate any occurrence of the character being used as the delimiter to another character before the record is checked for delimiters. So it is now possible to use any specific character as a delimiter and to be sure that the character never occurs in the data.  The User Guide for the Data Warehousing Utilities has been consolidated into the User Guide for the Instant Data Warehouse. This is possible because I have publicly published the User Guide for the Instant Data Warehouse. In the future, all my interesting comments on dimensional modelling and processing will be included into the User Guide for the Instant Data Warehouse. You can read the new 100+ page user guide by going to my Downloads Page. Press Release From the desk of Peter Nolan As you can see, all these upgrades have been around the area of loading data into a staging area prior to loading it into the Data Warehouse. The Instant Data Warehouse is extending it's reach so that customers can more easily load data into the Data Warehouse. Best Regards Peter Nolan