Data Warehouse Management March 13, 2000 Prof. Hwan-Seung Yong Dept. of CSE, Ewha Womans Univ. The Case for Data Warehousing The Case Against Data Warehousing Data Warehousing Gotchas Data Warehousing Software Evaluation

2000/3/9 H.S. Yong, Ewha Womans Univ.2 Basic Reason for Data Warehousing In Text –To convert data into business intelligence –make management decision making based on facts not intuition –get closer to the customers –gain competitive advantage But –data warehousing is only one step out of many in the long road toward the ultimate goal of accomplishing these highfalutin objectives More practical reason is in next slide

2000/3/9 H.S. Yong, Ewha Womans Univ.3 The Case for Data Warehousing To perform querying and reporting on servers/disks not used by OLTP systems To use data models and/or server technologies that speed up querying and reporting and that are not appropriate for transaction processing To provide an environment where a relatively small amount of knowledge of the technical aspects of database technology is required to write and maintain queries and reports and/or to provide a means to speed up the writing and maintaining of queries and reports by technical personnel To provide a repository of "cleaned up" transaction processing systems data that can be reported against and that does not necessarily require fixing the transaction processing systems

2000/3/9 H.S. Yong, Ewha Womans Univ.4 The Case for Data Warehousing To make it easier, to query and report data from multiple transaction processing systems and/or from external data sources To prevent persons who only need to query and report transaction processing system data from having any access whatsoever to transaction processing system databases and logic used to maintain those databases –security issues

2000/3/9 H.S. Yong, Ewha Womans Univ.5 The Case Against Data Warehousing Data warehousing systems, for the most part, store historical data that have been generated in internal transaction processing systems. This is a small part of the universe of data available to manage a business. Sometimes this part has limited value. Data warehousing systems can complicate business processes significantly. If most of your business needs are to report on data in one transaction processing system and/or all the historical data you need are in that system and/or the data in the system are clean and/or your hardware can support reporting against the live system data and/or the structure of the system data is relatively simple and/or your firm does not have much interest in end user ad hoc query/report tools, data warehousing may not be for your business. Data warehousing can have a learning curve that may be too long for impatient firms.

2000/3/9 H.S. Yong, Ewha Womans Univ.6 The Case Against Data Warehousing Data warehousing can become an exercise in data for the sake of the data. In certain organizations ad hoc end user query/reporting tools do not "take". Many "strategic applications" of data warehousing have a short life span and require the developers to put together a technically inelegant system quickly. Some developers are reluctant to work this way There is a limited number of people available who have worked with the full data warehousing system project "life cycle". Data warehousing systems can require a great deal of "maintenance" which many organizations cannot or will not support Sometimes the cost to capture data, clean it up, and deliver it in a format and time frame that is useful for the end users is too much of a cost to bear.

2000/3/9 H.S. Yong, Ewha Womans Univ.7 Data Warehousing Gotchas You are going to spend much time extracting, cleaning, and loading. Despite best efforts at project management, data warehousing project scope will increase. You are going to find problems with systems feeding the data warehouse. You will find the need to store data not being captured by any existing system Some transaction processing systems feeding the warehousing system will not contain detail You will underbudget for the resources skilled in the feeder system platforms Many warehouse end users will be trained and never or seldom apply their training

2000/3/9 H.S. Yong, Ewha Womans Univ.8 Data Warehousing Gotchas After end users receive query and report tools, requests for IS written reports may increase Your warehouse users will develop conflicting business rules Large scale data warehousing can become an exercise in data homogenizing 'Overhead' can eat up great amounts of disk space by precomputation The time it takes to load the warehouse will expand to the amount of the time in the available time window You are going to have a tough problem with security - especially if you make your data warehouse Web-accessible You will fail if you concentrate on resource optimization to the neglect of project, data, and customer management issues and an understanding of what adds value to the customer

2000/3/9 H.S. Yong, Ewha Womans Univ.9 Data Warehousing Software Evaluation Do the evaluation yourself –do not rely solely on the ideas of someone outside your organization –you know better than any outsider, your organization’s needs, expectations, limitations, resources Always first ask whether technology already in-house can do the job Get references –Ask the software vendor for a complete list of referenceable sites –If this is a major decision for your company, call 5-6 sites If you are going to see multiple vendor demos, build a test case that each vendor will follow Go through the site to find published evaluations of the software Be skeptical of data warehousing pundits' endorsements or reviews of technology

2000/3/9 H.S. Yong, Ewha Womans Univ.10 Data Warehousing Software Evaluation Go to the vendor road shows to talk with other attendees Understand the tradeoffs the software makes –tradeoff speed, capacity, computer resource consumption, ease of development, ease of use, and ease of maintenance Check the financial stability of the vendor Have a representative team perform the evaluation If you're evaluating an end user tool, let an end user lead the evaluation effort