Presentation is loading. Please wait.

Presentation is loading. Please wait.

Sybase Update >Joe Shaffner >Regional Technical Manager

Similar presentations


Presentation on theme: "Sybase Update >Joe Shaffner >Regional Technical Manager"— Presentation transcript:

1

2 Sybase Update >Joe Shaffner >Regional Technical Manager

3 Agenda >The Challenges of Data Warehousing >Sybase’s Approach >Recent Proof Points >Customer Examples / Success Stories

4 Agenda >Introduction >Traditional Approaches to Data Warehousing >Sybase Approach >Data Archiving

5 Data to Information to Knowledge >There is no shortage of information but there is a shortage of useful information >Data enables an enterprise to record an event >Information enables an enterprise to respond to an event >Knowledge enables an enterprise to anticipate an event

6 The Dimensions of Liquidity Flow “The data is there, I just can’t access it, can’t get to it.” “The data is locked up in silos.” Quality “I don’t always have the right data to make a decision” “The data isn’t presented in the right context that I can use.” Speed “I need the information in minutes not months.” “I have to wait until I’m back in the office to get data.”

7 BI System Purposes and Functions DM Review, April 2001 More ad hoc use than ever!

8 Why Do You to do Data Warehousing on the Fly? >Saving Money and time is what its all about now. >Decision cycles have significantly compressed Need answers now, not tomorrow. Each answer creates a new question Questions are ad hoc by nature >Project timelines to build a data warehouse have been reduced

9 Lots of Queries, Lots of Users, lots of Data -- Query displays service types by month -- Valid values are for month_key select month_key,service_key,count(*) from telco_facts where month_key = 1 group by month_key,service_key ; -- A look at customers who have the following service -- call waiting, caller id, and voice mail by fiscal period -- i.e. Q1,Q2,Q3,Q4 for year= 1998 select service_key,fiscal_period,count(*) from telco_facts T,month M where T.month_key=M.month_key and service_key = 4 group by fiscal_period,service_key order by fiscal_period,service_key ; -- female Customers in Massachusetts that do not have -- caller id select distinct(C.customer_key), C.customer_first_name, C.customer_last_name, C.phone_number from residential_customer C,service,telco_facts where C.customer_key = telco_facts.customer_key and telco_facts.service_key = service.service_key and caller_id_flag = 'N' and state = 'MA' and customer_gender = 'F' ; -- Find prospects for voice mail based on the criteria -- thatcustomers with call waiting and caller id are -- good prospects for call Waiting select state, count(*) from telco_facts T,service S, residential_customer C, month M where T.service_key = S.service_key and T.customer_key = C.customer_key and T.month_key = T.month_key and call_waiting_flag = 'Y' and caller_id_flag = 'Y' and voice_mail_flag = 'N' and state in ('NY','NJ','PA') and fiscal_period = 'Q1' group by state ; -- Find customers that had ISDN service in February 1998 selectcustomer_last_name,customer_first_name fromresidential_customer R,telco_facts T,service S,month M whereM.month_text = 'February ' andM.year = 1998 andS.isdn_flag = 'Y' andM.month_key = T.month_key andS.service_key = T.service_key andR.customer_key = T.customer_key ; -- Look at the local call minutes to see if they -- have increased after adding call waiting select fiscal_period,count(*),sum(local_call_minutes) from residential_customer R,telco_facts T,status S,month M where S.call_waiting_status='Added' and state = 'OH' and M.month_key= T.month_key and S.status_key = T.service_key and R.customer_key = T.customer_key group by fiscal_period order by fiscal_period ; -- Look at the call usage for customers with call waiting -- "service type 2" compared with customers with both call -- waiting and Caller id for Q4 for customers in CA select fiscal_period,T.service_key,sum(local_call_minutes), sum(local_call_count),count(*) from telco_facts T,residential_customer C,service S,month M where T.customer_key = C.customer_key and T.service_key = S.service_key and T.month_key = M.month_key and fiscal_period = 'Q4' and T.service_key in (02,03) and state = 'CA' group by fiscal_period,T.service_key ; -- Query displays service types by month -- Valid values are for month_key select month_key,service_key,count(*) from telco_facts where month_key = 1 group by month_key,service_key ; -- A look at customers who have the following service -- call waiting, caller id, and voice mail by fiscal period -- i.e. Q1,Q2,Q3,Q4 for year= 1998 select service_key,fiscal_period,count(*) from telco_facts T,month M where T.month_key=M.month_key and service_key = 4 group by fiscal_period,service_key order by fiscal_period,service_key ; -- female Customers in Massachusetts that do not have -- caller id select distinct(C.customer_key), C.customer_first_name, C.customer_last_name, C.phone_number from residential_customer C,service,telco_facts where C.customer_key = telco_facts.customer_key and telco_facts.service_key = service.service_key and caller_id_flag = 'N' and state = 'MA' and customer_gender = 'F' ; -- Find prospects for voice mail based on the criteria -- thatcustomers with call waiting and caller id are -- good prospects for call Waiting select state, count(*) from telco_facts T,service S, residential_customer C, month M where T.service_key = S.service_key and T.customer_key = C.customer_key and T.month_key = T.month_key and call_waiting_flag = 'Y' and caller_id_flag = 'Y' and voice_mail_flag = 'N' and state in ('NY','NJ','PA') and fiscal_period = 'Q1' group by state ; -- Find customers that had ISDN service in February 1998 selectcustomer_last_name,customer_first_name fromresidential_customer R,telco_facts T,service S,month M whereM.month_text = 'February ' andM.year = 1998 andS.isdn_flag = 'Y' andM.month_key = T.month_key andS.service_key = T.service_key andR.customer_key = T.customer_key ; -- Look at the local call minutes to see if they -- have increased after adding call waiting select fiscal_period,count(*),sum(local_call_minutes) from residential_customer R,telco_facts T,status S,month M where S.call_waiting_status='Added' and state = 'OH' and M.month_key= T.month_key and S.status_key = T.service_key and R.customer_key = T.customer_key group by fiscal_period order by fiscal_period ; -- Look at the call usage for customers with call waiting -- "service type 2" compared with customers with both call -- waiting and Caller id for Q4 for customers in CA select fiscal_period,T.service_key,sum(local_call_minutes), sum(local_call_count),count(*) from telco_facts T,residential_customer C,service S,month M where T.customer_key = C.customer_key and T.service_key = S.service_key and T.month_key = M.month_key and fiscal_period = 'Q4' and T.service_key in (02,03) and state = 'CA' group by fiscal_period,T.service_key ; -- Query displays service types by month -- Valid values are for month_key select month_key,service_key,count(*) from telco_facts where month_key = 1 group by month_key,service_key ; -- A look at customers who have the following service -- call waiting, caller id, and voice mail by fiscal period -- i.e. Q1,Q2,Q3,Q4 for year= 1998 select service_key,fiscal_period,count(*) from telco_facts T,month M where T.month_key=M.month_key and service_key = 4 group by fiscal_period,service_key order by fiscal_period,service_key ; -- female Customers in Massachusetts that do not have -- caller id select distinct(C.customer_key), C.customer_first_name, C.customer_last_name, C.phone_number from residential_customer C,service,telco_facts where C.customer_key = telco_facts.customer_key and telco_facts.service_key = service.service_key and caller_id_flag = 'N' and state = 'MA' and customer_gender = 'F' ; -- Find prospects for voice mail based on the criteria -- thatcustomers with call waiting and caller id are -- good prospects for call Waiting select state, count(*) from telco_facts T,service S, residential_customer C, month M where T.service_key = S.service_key and T.customer_key = C.customer_key and T.month_key = T.month_key and call_waiting_flag = 'Y' and caller_id_flag = 'Y' and voice_mail_flag = 'N' and state in ('NY','NJ','PA') and fiscal_period = 'Q1' group by state ; -- Find customers that had ISDN service in February 1998 selectcustomer_last_name,customer_first_name fromresidential_customer R,telco_facts T,service S,month M whereM.month_text = 'February ' andM.year = 1998 andS.isdn_flag = 'Y' andM.month_key = T.month_key andS.service_key = T.service_key andR.customer_key = T.customer_key ; -- Look at the local call minutes to see if they -- have increased after adding call waiting select fiscal_period,count(*),sum(local_call_minutes) from residential_customer R,telco_facts T,status S,month M where S.call_waiting_status='Added' and state = 'OH' and M.month_key= T.month_key and S.status_key = T.service_key and R.customer_key = T.customer_key group by fiscal_period order by fiscal_period ; -- Look at the call usage for customers with call waiting -- "service type 2" compared with customers with both call -- waiting and Caller id for Q4 for customers in CA select fiscal_period,T.service_key,sum(local_call_minutes), sum(local_call_count),count(*) from telco_facts T,residential_customer C,service S,month M where T.customer_key = C.customer_key and T.service_key = S.service_key and T.month_key = M.month_key and fiscal_period = 'Q4' and T.service_key in (02,03) and state = 'CA' group by fiscal_period,T.service_key ; -- Query displays service types by month -- Valid values are for month_key select month_key,service_key,count(*) from telco_facts where month_key = 1 group by month_key,service_key ; -- A look at customers who have the following service -- call waiting, caller id, and voice mail by fiscal period -- i.e. Q1,Q2,Q3,Q4 for year= 1998 select service_key,fiscal_period,count(*) from telco_facts T,month M where T.month_key=M.month_key and service_key = 4 group by fiscal_period,service_key order by fiscal_period,service_key ; -- female Customers in Massachusetts that do not have -- caller id select distinct(C.customer_key), C.customer_first_name, C.customer_last_name, C.phone_number from residential_customer C,service,telco_facts where C.customer_key = telco_facts.customer_key and telco_facts.service_key = service.service_key and caller_id_flag = 'N' and state = 'MA' and customer_gender = 'F' ; -- Find prospects for voice mail based on the criteria -- thatcustomers with call waiting and caller id are -- good prospects for call Waiting select state, count(*) from telco_facts T,service S, residential_customer C, month M where T.service_key = S.service_key and T.customer_key = C.customer_key and T.month_key = T.month_key and call_waiting_flag = 'Y' and caller_id_flag = 'Y' and voice_mail_flag = 'N' and state in ('NY','NJ','PA') and fiscal_period = 'Q1' group by state ; -- Find customers that had ISDN service in February 1998 selectcustomer_last_name,customer_first_name fromresidential_customer R,telco_facts T,service S,month M whereM.month_text = 'February ' andM.year = 1998 andS.isdn_flag = 'Y' andM.month_key = T.month_key andS.service_key = T.service_key andR.customer_key = T.customer_key ; -- Look at the local call minutes to see if they -- have increased after adding call waiting select fiscal_period,count(*),sum(local_call_minutes) from residential_customer R,telco_facts T,status S,month M where S.call_waiting_status='Added' and state = 'OH' and M.month_key= T.month_key and S.status_key = T.service_key and R.customer_key = T.customer_key group by fiscal_period order by fiscal_period ; -- Look at the call usage for customers with call waiting -- "service type 2" compared with customers with both call -- waiting and Caller id for Q4 for customers in CA select fiscal_period,T.service_key,sum(local_call_minutes), sum(local_call_count),count(*) from telco_facts T,residential_customer C,service S,month M where T.customer_key = C.customer_key and T.service_key = S.service_key and T.month_key = M.month_key and fiscal_period = 'Q4' and T.service_key in (02,03) and state = 'CA' group by fiscal_period,T.service_key ; -- Query displays service types by month -- Valid values are for month_key select month_key,service_key,count(*) from telco_facts where month_key = 1 group by month_key,service_key ; -- A look at customers who have the following service -- call waiting, caller id, and voice mail by fiscal period -- i.e. Q1,Q2,Q3,Q4 for year= 1998 select service_key,fiscal_period,count(*) from telco_facts T,month M where T.month_key=M.month_key and service_key = 4 group by fiscal_period,service_key order by fiscal_period,service_key ; -- female Customers in Massachusetts that do not have -- caller id select distinct(C.customer_key), C.customer_first_name, C.customer_last_name, C.phone_number from residential_customer C,service,telco_facts where C.customer_key = telco_facts.customer_key and telco_facts.service_key = service.service_key and caller_id_flag = 'N' and state = 'MA' and customer_gender = 'F' ; -- Find prospects for voice mail based on the criteria -- thatcustomers with call waiting and caller id are -- good prospects for call Waiting select state, count(*) from telco_facts T,service S, residential_customer C, month M where T.service_key = S.service_key and T.customer_key = C.customer_key and T.month_key = T.month_key and call_waiting_flag = 'Y' and caller_id_flag = 'Y' and voice_mail_flag = 'N' and state in ('NY','NJ','PA') and fiscal_period = 'Q1' group by state ; -- Find customers that had ISDN service in February 1998 selectcustomer_last_name,customer_first_name fromresidential_customer R,telco_facts T,service S,month M whereM.month_text = 'February ' andM.year = 1998 andS.isdn_flag = 'Y' andM.month_key = T.month_key andS.service_key = T.service_key andR.customer_key = T.customer_key ; -- Look at the local call minutes to see if they -- have increased after adding call waiting select fiscal_period,count(*),sum(local_call_minutes) from residential_customer R,telco_facts T,status S,month M where S.call_waiting_status='Added' and state = 'OH' and M.month_key= T.month_key and S.status_key = T.service_key and R.customer_key = T.customer_key group by fiscal_period order by fiscal_period ; -- Look at the call usage for customers with call waiting -- "service type 2" compared with customers with both call -- waiting and Caller id for Q4 for customers in CA select fiscal_period,T.service_key,sum(local_call_minutes), sum(local_call_count),count(*) from telco_facts T,residential_customer C,service S,month M where T.customer_key = C.customer_key and T.service_key = S.service_key and T.month_key = M.month_key and fiscal_period = 'Q4' and T.service_key in (02,03) and state = 'CA' group by fiscal_period,T.service_key ; -- Query displays service types by month -- Valid values are for month_key select month_key,service_key,count(*) from telco_facts where month_key = 1 group by month_key,service_key ; -- A look at customers who have the following service -- call waiting, caller id, and voice mail by fiscal period -- i.e. Q1,Q2,Q3,Q4 for year= 1998 select service_key,fiscal_period,count(*) from telco_facts T,month M where T.month_key=M.month_key and service_key = 4 group by fiscal_period,service_key order by fiscal_period,service_key ; -- female Customers in Massachusetts that do not have -- caller id select distinct(C.customer_key), C.customer_first_name, C.customer_last_name, C.phone_number from residential_customer C,service,telco_facts where C.customer_key = telco_facts.customer_key and telco_facts.service_key = service.service_key and caller_id_flag = 'N' and state = 'MA' and customer_gender = 'F' ; -- Find prospects for voice mail based on the criteria -- thatcustomers with call waiting and caller id are -- good prospects for call Waiting select state, count(*) from telco_facts T,service S, residential_customer C, month M where T.service_key = S.service_key and T.customer_key = C.customer_key and T.month_key = T.month_key and call_waiting_flag = 'Y' and caller_id_flag = 'Y' and voice_mail_flag = 'N' and state in ('NY','NJ','PA') and fiscal_period = 'Q1' group by state ; -- Find customers that had ISDN service in February 1998 selectcustomer_last_name,customer_first_name fromresidential_customer R,telco_facts T,service S,month M whereM.month_text = 'February ' andM.year = 1998 andS.isdn_flag = 'Y' andM.month_key = T.month_key andS.service_key = T.service_key andR.customer_key = T.customer_key ; -- Look at the local call minutes to see if they -- have increased after adding call waiting select fiscal_period,count(*),sum(local_call_minutes) from residential_customer R,telco_facts T,status S,month M where S.call_waiting_status='Added' and state = 'OH' and M.month_key= T.month_key and S.status_key = T.service_key and R.customer_key = T.customer_key group by fiscal_period order by fiscal_period ; -- Look at the call usage for customers with call waiting -- "service type 2" compared with customers with both call -- waiting and Caller id for Q4 for customers in CA select fiscal_period,T.service_key,sum(local_call_minutes), sum(local_call_count),count(*) from telco_facts T,residential_customer C,service S,month M where T.customer_key = C.customer_key and T.service_key = S.service_key and T.month_key = M.month_key and fiscal_period = 'Q4' and T.service_key in (02,03) and state = 'CA' group by fiscal_period,T.service_key ; -- Query displays service types by month -- Valid values are for month_key select month_key,service_key,count(*) from telco_facts where month_key = 1 group by month_key,service_key ; -- A look at customers who have the following service -- call waiting, caller id, and voice mail by fiscal period -- i.e. Q1,Q2,Q3,Q4 for year= 1998 select service_key,fiscal_period,count(*) from telco_facts T,month M where T.month_key=M.month_key and service_key = 4 group by fiscal_period,service_key order by fiscal_period,service_key ; -- female Customers in Massachusetts that do not have -- caller id select distinct(C.customer_key), C.customer_first_name, C.customer_last_name, C.phone_number from residential_customer C,service,telco_facts where C.customer_key = telco_facts.customer_key and telco_facts.service_key = service.service_key and caller_id_flag = 'N' and state = 'MA' and customer_gender = 'F' ; -- Find prospects for voice mail based on the criteria -- thatcustomers with call waiting and caller id are -- good prospects for call Waiting select state, count(*) from telco_facts T,service S, residential_customer C, month M where T.service_key = S.service_key and T.customer_key = C.customer_key and T.month_key = T.month_key and call_waiting_flag = 'Y' and caller_id_flag = 'Y' and voice_mail_flag = 'N' and state in ('NY','NJ','PA') and fiscal_period = 'Q1' group by state ; -- Find customers that had ISDN service in February 1998 selectcustomer_last_name,customer_first_name fromresidential_customer R,telco_facts T,service S,month M whereM.month_text = 'February ' andM.year = 1998 andS.isdn_flag = 'Y' andM.month_key = T.month_key andS.service_key = T.service_key andR.customer_key = T.customer_key ; -- Look at the local call minutes to see if they -- have increased after adding call waiting select fiscal_period,count(*),sum(local_call_minutes) from residential_customer R,telco_facts T,status S,month M where S.call_waiting_status='Added' and state = 'OH' and M.month_key= T.month_key and S.status_key = T.service_key and R.customer_key = T.customer_key group by fiscal_period order by fiscal_period ; -- Look at the call usage for customers with call waiting -- "service type 2" compared with customers with both call -- waiting and Caller id for Q4 for customers in CA select fiscal_period,T.service_key,sum(local_call_minutes), sum(local_call_count),count(*) from telco_facts T,residential_customer C,service S,month M where T.customer_key = C.customer_key and T.service_key = S.service_key and T.month_key = M.month_key and fiscal_period = 'Q4' and T.service_key in (02,03) and state = 'CA' group by fiscal_period,T.service_key ; -- Query displays service types by month -- Valid values are for month_key select month_key,service_key,count(*) from telco_facts where month_key = 1 group by month_key,service_key ; -- A look at customers who have the following service -- call waiting, caller id, and voice mail by fiscal period -- i.e. Q1,Q2,Q3,Q4 for year= 1998 select service_key,fiscal_period,count(*) from telco_facts T,month M where T.month_key=M.month_key and service_key = 4 group by fiscal_period,service_key order by fiscal_period,service_key ; -- female Customers in Massachusetts that do not have -- caller id select distinct(C.customer_key), C.customer_first_name, C.customer_last_name, C.phone_number from residential_customer C,service,telco_facts where C.customer_key = telco_facts.customer_key and telco_facts.service_key = service.service_key and caller_id_flag = 'N' and state = 'MA' and customer_gender = 'F' ; -- Find prospects for voice mail based on the criteria -- thatcustomers with call waiting and caller id are -- good prospects for call Waiting select state, count(*) from telco_facts T,service S, residential_customer C, month M where T.service_key = S.service_key and T.customer_key = C.customer_key and T.month_key = T.month_key and call_waiting_flag = 'Y' and caller_id_flag = 'Y' and voice_mail_flag = 'N' and state in ('NY','NJ','PA') and fiscal_period = 'Q1' group by state ; -- Find customers that had ISDN service in February 1998 selectcustomer_last_name,customer_first_name fromresidential_customer R,telco_facts T,service S,month M whereM.month_text = 'February ' andM.year = 1998 andS.isdn_flag = 'Y' andM.month_key = T.month_key andS.service_key = T.service_key andR.customer_key = T.customer_key ; -- Look at the local call minutes to see if they -- have increased after adding call waiting select fiscal_period,count(*),sum(local_call_minutes) from residential_customer R,telco_facts T,status S,month M where S.call_waiting_status='Added' and state = 'OH' and M.month_key= T.month_key and S.status_key = T.service_key and R.customer_key = T.customer_key group by fiscal_period order by fiscal_period ; -- Look at the call usage for customers with call waiting -- "service type 2" compared with customers with both call -- waiting and Caller id for Q4 for customers in CA select fiscal_period,T.service_key,sum(local_call_minutes), sum(local_call_count),count(*) from telco_facts T,residential_customer C,service S,month M where T.customer_key = C.customer_key and T.service_key = S.service_key and T.month_key = M.month_key and fiscal_period = 'Q4' and T.service_key in (02,03) and state = 'CA' group by fiscal_period,T.service_key ; -- Query displays service types by month -- Valid values are for month_key select month_key,service_key,count(*) from telco_facts where month_key = 1 group by month_key,service_key ; -- A look at customers who have the following service -- call waiting, caller id, and voice mail by fiscal period -- i.e. Q1,Q2,Q3,Q4 for year= 1998 select service_key,fiscal_period,count(*) from telco_facts T,month M where T.month_key=M.month_key and service_key = 4 group by fiscal_period,service_key order by fiscal_period,service_key ; -- female Customers in Massachusetts that do not have -- caller id select distinct(C.customer_key), C.customer_first_name, C.customer_last_name, C.phone_number from residential_customer C,service,telco_facts where C.customer_key = telco_facts.customer_key and telco_facts.service_key = service.service_key and caller_id_flag = 'N' and state = 'MA' and customer_gender = 'F' ; -- Find prospects for voice mail based on the criteria -- thatcustomers with call waiting and caller id are -- good prospects for call Waiting select state, count(*) from telco_facts T,service S, residential_customer C, month M where T.service_key = S.service_key and T.customer_key = C.customer_key and T.month_key = T.month_key and call_waiting_flag = 'Y' and caller_id_flag = 'Y' and voice_mail_flag = 'N' and state in ('NY','NJ','PA') and fiscal_period = 'Q1' group by state ; -- Find customers that had ISDN service in February 1998 selectcustomer_last_name,customer_first_name fromresidential_customer R,telco_facts T,service S,month M whereM.month_text = 'February ' andM.year = 1998 andS.isdn_flag = 'Y' andM.month_key = T.month_key andS.service_key = T.service_key andR.customer_key = T.customer_key ; -- Look at the local call minutes to see if they -- have increased after adding call waiting select fiscal_period,count(*),sum(local_call_minutes) from residential_customer R,telco_facts T,status S,month M where S.call_waiting_status='Added' and state = 'OH' and M.month_key= T.month_key and S.status_key = T.service_key and R.customer_key = T.customer_key group by fiscal_period order by fiscal_period ; -- Look at the call usage for customers with call waiting -- "service type 2" compared with customers with both call -- waiting and Caller id for Q4 for customers in CA select fiscal_period,T.service_key,sum(local_call_minutes), sum(local_call_count),count(*) from telco_facts T,residential_customer C,service S,month M where T.customer_key = C.customer_key and T.service_key = S.service_key and T.month_key = M.month_key and fiscal_period = 'Q4' and T.service_key in (02,03) and state = 'CA' group by fiscal_period,T.service_key ; -- Query displays service types by month -- Valid values are for month_key select month_key,service_key,count(*) from telco_facts where month_key = 1 group by month_key,service_key ; -- A look at customers who have the following service -- call waiting, caller id, and voice mail by fiscal period -- i.e. Q1,Q2,Q3,Q4 for year= 1998 select service_key,fiscal_period,count(*) from telco_facts T,month M where T.month_key=M.month_key and service_key = 4 group by fiscal_period,service_key order by fiscal_period,service_key ; -- female Customers in Massachusetts that do not have -- caller id select distinct(C.customer_key), C.customer_first_name, C.customer_last_name, C.phone_number from residential_customer C,service,telco_facts where C.customer_key = telco_facts.customer_key and telco_facts.service_key = service.service_key and caller_id_flag = 'N' and state = 'MA' and customer_gender = 'F' ; -- Find prospects for voice mail based on the criteria -- thatcustomers with call waiting and caller id are -- good prospects for call Waiting select state, count(*) from telco_facts T,service S, residential_customer C, month M where T.service_key = S.service_key and T.customer_key = C.customer_key and T.month_key = T.month_key and call_waiting_flag = 'Y' and caller_id_flag = 'Y' and voice_mail_flag = 'N' and state in ('NY','NJ','PA') and fiscal_period = 'Q1' group by state ; -- Find customers that had ISDN service in February 1998 selectcustomer_last_name,customer_first_name fromresidential_customer R,telco_facts T,service S,month M whereM.month_text = 'February ' andM.year = 1998 andS.isdn_flag = 'Y' andM.month_key = T.month_key andS.service_key = T.service_key andR.customer_key = T.customer_key ; -- Look at the local call minutes to see if they -- have increased after adding call waiting select fiscal_period,count(*),sum(local_call_minutes) from residential_customer R,telco_facts T,status S,month M where S.call_waiting_status='Added' and state = 'OH' and M.month_key= T.month_key and S.status_key = T.service_key and R.customer_key = T.customer_key group by fiscal_period order by fiscal_period ; -- Look at the call usage for customers with call waiting -- "service type 2" compared with customers with both call -- waiting and Caller id for Q4 for customers in CA select fiscal_period,T.service_key,sum(local_call_minutes), sum(local_call_count),count(*) from telco_facts T,residential_customer C,service S,month M where T.customer_key = C.customer_key and T.service_key = S.service_key and T.month_key = M.month_key and fiscal_period = 'Q4' and T.service_key in (02,03) and state = 'CA' group by fiscal_period,T.service_key ; -- Query displays service types by month -- Valid values are for month_key select month_key,service_key,count(*) from telco_facts where month_key = 1 group by month_key,service_key ; -- A look at customers who have the following service -- call waiting, caller id, and voice mail by fiscal period -- i.e. Q1,Q2,Q3,Q4 for year= 1998 select service_key,fiscal_period,count(*) from telco_facts T,month M where T.month_key=M.month_key and service_key = 4 group by fiscal_period,service_key order by fiscal_period,service_key ; -- female Customers in Massachusetts that do not have -- caller id select distinct(C.customer_key), C.customer_first_name, C.customer_last_name, C.phone_number from residential_customer C,service,telco_facts where C.customer_key = telco_facts.customer_key and telco_facts.service_key = service.service_key and caller_id_flag = 'N' and state = 'MA' and customer_gender = 'F' ; -- Find prospects for voice mail based on the criteria -- thatcustomers with call waiting and caller id are -- good prospects for call Waiting select state, count(*) from telco_facts T,service S, residential_customer C, month M where T.service_key = S.service_key and T.customer_key = C.customer_key and T.month_key = T.month_key and call_waiting_flag = 'Y' and caller_id_flag = 'Y' and voice_mail_flag = 'N' and state in ('NY','NJ','PA') and fiscal_period = 'Q1' group by state ; -- Find customers that had ISDN service in February 1998 selectcustomer_last_name,customer_first_name fromresidential_customer R,telco_facts T,service S,month M whereM.month_text = 'February ' andM.year = 1998 andS.isdn_flag = 'Y' andM.month_key = T.month_key andS.service_key = T.service_key andR.customer_key = T.customer_key ; -- Look at the local call minutes to see if they -- have increased after adding call waiting select fiscal_period,count(*),sum(local_call_minutes) from residential_customer R,telco_facts T,status S,month M where S.call_waiting_status='Added' and state = 'OH' and M.month_key= T.month_key and S.status_key = T.service_key and R.customer_key = T.customer_key group by fiscal_period order by fiscal_period ; -- Look at the call usage for customers with call waiting -- "service type 2" compared with customers with both call -- waiting and Caller id for Q4 for customers in CA select fiscal_period,T.service_key,sum(local_call_minutes), sum(local_call_count),count(*) from telco_facts T,residential_customer C,service S,month M where T.customer_key = C.customer_key and T.service_key = S.service_key and T.month_key = M.month_key and fiscal_period = 'Q4' and T.service_key in (02,03) and state = 'CA' group by fiscal_period,T.service_key ; -- Query displays service types by month -- Valid values are for month_key select month_key,service_key,count(*) from telco_facts where month_key = 1 group by month_key,service_key ; -- A look at customers who have the following service -- call waiting, caller id, and voice mail by fiscal period -- i.e. Q1,Q2,Q3,Q4 for year= 1998 select service_key,fiscal_period,count(*) from telco_facts T,month M where T.month_key=M.month_key and service_key = 4 group by fiscal_period,service_key order by fiscal_period,service_key ; -- female Customers in Massachusetts that do not have -- caller id select distinct(C.customer_key), C.customer_first_name, C.customer_last_name, C.phone_number from residential_customer C,service,telco_facts where C.customer_key = telco_facts.customer_key and telco_facts.service_key = service.service_key and caller_id_flag = 'N' and state = 'MA' and customer_gender = 'F' ; -- Find prospects for voice mail based on the criteria -- thatcustomers with call waiting and caller id are -- good prospects for call Waiting select state, count(*) from telco_facts T,service S, residential_customer C, month M where T.service_key = S.service_key and T.customer_key = C.customer_key and T.month_key = T.month_key and call_waiting_flag = 'Y' and caller_id_flag = 'Y' and voice_mail_flag = 'N' and state in ('NY','NJ','PA') and fiscal_period = 'Q1' group by state ; -- Find customers that had ISDN service in February 1998 selectcustomer_last_name,customer_first_name fromresidential_customer R,telco_facts T,service S,month M whereM.month_text = 'February ' andM.year = 1998 andS.isdn_flag = 'Y' andM.month_key = T.month_key andS.service_key = T.service_key andR.customer_key = T.customer_key ; -- Look at the local call minutes to see if they -- have increased after adding call waiting select fiscal_period,count(*),sum(local_call_minutes) from residential_customer R,telco_facts T,status S,month M where S.call_waiting_status='Added' and state = 'OH' and M.month_key= T.month_key and S.status_key = T.service_key and R.customer_key = T.customer_key group by fiscal_period order by fiscal_period ; -- Look at the call usage for customers with call waiting -- "service type 2" compared with customers with both call -- waiting and Caller id for Q4 for customers in CA select fiscal_period,T.service_key,sum(local_call_minutes), sum(local_call_count),count(*) from telco_facts T,residential_customer C,service S,month M where T.customer_key = C.customer_key and T.service_key = S.service_key and T.month_key = M.month_key and fiscal_period = 'Q4' and T.service_key in (02,03) and state = 'CA' group by fiscal_period,T.service_key ;

10 Traditional Row Based DBMS >Designed to support On-line transaction processing (OLTP) >GOOD at getting data in quickly assuring referential integrity >Not GOOD at: Getting data out quickly Supporting ad hoc queries >Requires table scans Storing data efficiently >Requires many indexes >May require pre-aggregations >Have been retro-fitted to support data warehouses >Still run into to the same “old” limitations Examples: MSSQL Sybase ASE Oracle 8.1 DB2

11 Traditional Data Warehouse Databases >Were designed for Data Warehousing >Are good at Ad Hoc Analysis Supporting many users Loading data >But are; Challenging to implement Costly to Maintain Expensive to Own May not scale well as users are added. Examples: Red Brick TeraData

12 The Sybase Approach

13 ASIQ – What is it? >Adaptive Server IQ is a Relational DB System >developed in 1993 >employs patented bitwise indexes for fast query response > Adaptive Server IQ was designed specifically for Data Warehousing > Has the look and feel of a typical RDBMS > But under the covers it bears little resemblance > 2,500 Customers Worldwide > Mission Critical applications > VLDB > IQ customers grew by 125% last year

14 Traditional Solution - Parallelism > 800 Bytes per row > 16K Page Size > 20 Million Rows > 500,000 I/O’s Needed > Parallelism won’t solve this problem >More Hardware, slightly faster >Only 1 Query can run at a time >Very expensive and inflexible to ad-hoc queries Calculate average sales for “A” stores in New York Calculate average sales for “A” stores in New York

15 ASIQ Architectural Summary Read Full Row Read Relevant Columns Read Bitmap  Bitmap structures are built on all fields  Bitmaps further reduce the amount of data read  Small number of bits rather than entire field  ANDing and ORing bitmaps is very efficient with today’s processors  Note that even vertically stored data is not read  Data structure storage and manipulation is transparent to applications and administrators. New values are handled automatically.

16 Distinctive Advantages of IQ  Storage Efficiency  20-50% of raw data vs %  Query Efficiency  x faster than traditional RDBMS  Database Loading  Load & Index while reading  Scalability  Billions of rows, thousands of users, hundreds of nodes  3 million internet users  Disk Input-Output Efficiency  60x less I/O than traditional DBMS  Simple to Administer  DBA load is 25% of traditional DBMS  Any schema (Multidimensional, etc.) IQ Data Store “IQ deserves the attention of ANYONE evaluating data warehouse DBMS options.” Rich Finkelstein, Performance Computing Inc.

17 Gartner Measurement - Amount of Detailed Data Managing Large Amounts of Detailed Data LOAD INPUT DATA: 1TB -Source: flat files, ETL, replication,ODS Base table(FP): TB Indexes: TB Aggr/Summ: 0-0.1TB Summaries Aggregates 1-2 TB Indexes 0.5-3TB Base table (“RAW data”) (no indexes) TB TB IQ Multiplex Conventional DBMS Same INPUT Data: “Conventional DW” is 3x-10x larger than IQ-M DW TB

18 Source: Meta Group 2/98 Elements of TCO for DW “Over a 5 year period the cost of managing a data warehouse is typically 3X the initial budget.” Hardware consumes a large portion of the budget

19 BI Users: How Many Is That??

20 ASIQ Multiplex – What Is It? >Version of ASIQ >Purpose is to Extend ASIQ Scalability >Extends Single Database Access across Multiple Computer Nodes >Allows Mixing of Unix and NT Nodes >NOT an MPP Solution >Much Simpler Implementation >Much Simpler Management

21 You build your Warehouse … IQ It’s successful ! It’s very successful ! It’s too successful !!!!!! At some point you hit the wall with performance

22 ASIQ Multiplex – What Is IT? >Version of ASIQ >Purpose is to Extend ASIQ Scalability >Extends Single Database Access across Multiple Computer Nodes >NOT an MPP Solution >Much Simpler Implementation >Much Simpler Management

23 IQ Multiplex Configuration Each IQ Node has its own: - CPU’s - Local Temp Space (Disk) - Memory - Catalog All data and indexes are stored in the shared IQ database, which is on fiber channel or EMC type storage systems Individual nodes can be different configurations (CPUs, memory, disk) IQ Unix Server IQ Unix Server IQ Unix IQ Multiplex functions Unix Server IQ Unix IQ Multiplex functions Unix Server IQ Unix IQ Multiplex functions

24 Scaling to More Users or More Data IQ Multiplex functions Unix Server IQ Unix IQ Unix Server IQ Unix Server IQ Unix VLM Unix Server IQ Unix IQ Write Node Read Only Nodes Unix Server IQ Unix Server IQ Unix Server IQ Unix No Data Redistribution No Change in Schema Replication of Catalog for Logins Very little I/O contention (1/10 of Oracle Parallel Server) Read Only Nodes

25 Enterprise Entrepreneurial VLDB Web Portals Adaptive Server IQ Multiplex Exclusive Prepackaged Scalability Adaptive Server IQ Multiplex 10’s of Users 100’s of Users 1000’s of Users

26 Adaptive Server IQ Index Types: >Low-Fast(LF) This index is ideal for columns that have a very low number of unique values (under 1000) Example: Gender, Yes/No, State >High_Group (HG) Typically used to process equality and group by operations on high-cardinality data (recommended for more than 1000 distinct values) >High_Non_Group (HNG) Add an HNG index when you need to do range searches. The number of unique values is high (greater than 1000) You don't need to do GROUP BY on the column >Compare (CMP) A Compare (CMP) index is an index on the relationship between two columns

27 Adaptive Server IQ Index Types: >Containment (WD) This index allows you to store words from a column string of CHAR and VARCHAR data. Use a WD index for the fastest access to columns that contain a list of keywords (for example, in bibliographic record or Web page). > The Date (DATE), Time (TIME), and Datetime (DTTM) Three index types are used to process queries involving date, time, or datetime quantities: A DATE index is used on columns of data type DATE to process certain queries involving date quantities. The TIME index is used on columns of data type TIME to process certain queries involving time quantities. The DTTM index is used on columns of data type DATETIME or TIMESTAMP to process certain queries involving datetime quantities.

28 IQ Hardware and Software Partners >PLATFORMS >Sun Solaris >HP - UX >IBM AIX (Simplex) >NT >LINUX 32 bit >WEB ANALYSIS APPL. >Compudigm >Web Trends >Informatica >CRM ANALYSIS APPL. >Industry Warehouse Studio * >ADVANCED VISUALIZATION >Compudigm >ETL / DATA MOVEMENT >Informatica * >Sybase Replication Server >WEB REPORTING APPL. >Actuate e.Reporting Suite

29 IQ Hardware and Software Partners >ANALYSIS TOOLS >Business Objects >Cognos >Brio >Micro Strategy >Easy Ask >Whitelight >Hummingbird >ANALYSIS TOOLS >MS Access >SAS >Group 1 >MineSet (SGI) >PowerDesigner * >Warehouse Control Center * >PowerBuilder *

30 OLAP Tools And IQ: A Beautiful Thing >Demonstrates IQ well >OLAP Tools do ad hoc queries >OLAP Tools can bring a traditional RDBMS to its knees. >OLAP tools can be used by large numbers of people >The success of a Data Warehouse is largely determined by its acceptance by end users >If the OLAP tool becomes popular More data will be requested More users

31 Late Breaking News

32 The Sybase Approach

33 References : Amount of Detailed Data Nielsen Media Research Business Issues Leader in TV ratings business 5 to 10 years of TV viewer history Cost and logistics becoming costly Need same data on multiple databases to scale Difficult to deliver new services to customers Results Sun/Sybase delivered Reference Architecture 35 TB benchmark and best practices guide 12 TB detailed input data in production Fast access and data load Linear scalability to 108 CPUs Architected for 100s of TB on Hitachi SNA “The big advantage of the Sun Sybase Reference Architecture is that it provides the advance knowledge that this solution will work.” “We are able to deliver one data warehouse for all our applications, at one-third the storage of conventional technologies, while seeing performance gains as advertised with IQ Multiplex.” Kim Ross CIO Nielsen Media Research

34 References : Query Complexity U.S. DOT: Bureau of Transportation Statistics Business Issues Congressional legislation required consolidated, single-point of access to all transportation statistics Needed to deliver over the Web Over 250+ databases of source data Results 2.5TB of detailed input data compressed to 1 TB Query complexity with 18-way joins Reduced data gathering time Easy linkages across many data sets allows new insights on transportation safety The new website is aimed at transportation researchers and analysts Website gets 15,000 hits per day Sybase IQ reduced loading and indexing from 30 minutes to 2.5 to 3 minutes. Query speeds were 20 – 50 times faster than Oracle. Time to add a column was reduced from 4 hours with Oracle to 15 minutes with IQ. Jeff Butler Assistant Director, Office of Statistical Computing Department of Transportation Bureau of Transportation Statistics

35 BizRate gains Economical Data Management with Sun and Sybase Together, Sun and Sybase have created a solution that packs an extraordinary amount of data processing and analytical power into a small footprint that represents a realistic investment for small and mid-sized firms. Sybase’s tight architectural integration with Sun technology provides us with the assurance we need that the technology foundation of our data warehouse will scale to meet our growing needs in the future. Henri Asseily, Chief Technology Officer and Founder of BizRate.com Business Issues >Delivers analysis of internet utilization >Leading online customer survey producer >Cost and logistics becoming unwieldy >Microsoft SQL Server could not scale >Simplify data deliver and analysis for sellers Business Results >Manage 15 million customer data sets >Tight integration through Reference Architecture >Scaleable solution that will grow >Delivered on Sun Fire V880

36 References : Amount of Detailed Data & Query Complexity Internal Revenue Service Business Issues Analysis virtually impossible Lost productivity Loss of potential billions in revenue VLDB management Results 10+ TB detailed input data (10 yrs of taxpayer records) fits in 5 TB of storage Query complexity with 14-way joins Average 120 ad-hoc analysis users Modeled entire population of commercial tax returns Supports advanced analysis like data mining Revenue protection & fraud detection ROI of $250 Million Portal-enabled “The primary technology challenge was to build a system that could manage such large volumes of data and yet was sufficiently open to facilitate queries from various off-the-shelf products. We selected Sybase ASIQ as the data-management server, based on its strength with decision support type queries.” Jeff Kmonk Manager, Office of Research Compliance Research Division Internal Revenue Service

37 References : Amount of Data and # of Concurrent Users NC: Dept. of Health and Human Services Business Issues System required to provide information for federally mandated reports. Also used for fraud detection for USDA and food stamp programs Results Now serving 1,200 users on 4.5TB, to grow to 28TB Recently used to uncover $3.5 million in Medicaid claims saving Approx. $18 million in storage savings Non-disclosure reference

38 References : Amount of Detailed Data & Number of Users & Complex Data Models American Express Global Fraud Detection Business Issues Unable to perform advanced analysis of fraud patterns for credit card transactions with competing solutions due to performance issues, query complexity limitations Needed solutions to handle 700-column table to describe every transaction, at least one year of transactions online Results Advanced fraud analysis possible for last 4 years Over 1,600 users worldwide 6TB of input data; 10 billion records (last 13 months) of credit card transactions online All fraud managers worldwide use IQ Multiplex system in AZ Over 90% of database is fraud detection information Non-disclosure reference

39 References : Query Complexity Bank of Montreal Business Issues Identify and retain most profitable customers Increase effectiveness of marketing programs Attract new customers Access to multiple information systems and “touch points” Cutting edge technology and architecture Results 1 Terabyte data warehouse Avg. query complexity with 18-way joins IRR over 100% Average credit card volume up 59% Average credit card balances up 129% Market share up 60 basis points System up in 4 months Component architecture enables flexibility “We felt you should develop the data warehouse component by component because that allows you to apply what you learn.” “Sybase was truly committed to ensuring that we used technology in a way that really impacted the business.” Carl A. Touchie Sr. Manager Electronic Financial Services Bank of Montreal

40 Distinctive Advantages of IQ >Query Efficiency >10-100x faster than traditional RDBMS >Database Loading >600+ million records/300+ GB of data per week, 2+ billion rows a month >Scalability >Billions of rows, thousands of users, hundreds of nodes >3 million internet users >Storage Efficiency >20-70% of raw data vs % >Disk Input-Output Efficiency >60x less I/O than traditional DBMS >Simple to Administer >DBA load is 25% of traditional DBMS >Any schema (Multidimensional, etc.)

41 Recent Customer Examples

42 Overview for Large Southern Bank October 2 nd, 2003 Sybase IQ Proof of Concept

43 Your Requirements >Reduce EMC Storage costs >Simple and fast implementation >Faster query speeds

44 Who is using IQ today for Archiving? >Nielsen Media Loaded 10 years of detailed TV viewer patterns from mainframe archive.  f > U.S. Internal Revenue Service Keeps years ’ worth of all U.S tax returns on disk.  >EMI Music Germany Stores 10 years of historical data on and delivers responses to ad-hoc queries within seconds.  >North Carolina Department of Health and Human Services Recently used to uncover $3.5 million in Medicaid claims saving. Serving 1,200 Approx. $18 million in storage savings >2 TB of input/raw data, IQ-M: 1.5 TB, Recently used to uncover $3.5 million in Medicaid claims saving >Approx. $18 million in storage savings

45 TIME LINE DATE AUGUST SEPTEMBER 3rd 8 th 15th 16th OCTOBER 1st 2nd TASK Initial IQ Presentation Scoping Meeting Pre POC Preparations Installation, configuration, load Load completion Tabulation of Results (Two Days!) Complete data movement Run sample queries Presentation TODAY Stumbling Blocks 1.Dr Watson prevented final table load on day 1 2.Special Characters in data found in extract prevented load completion

46 Current Environment BCP ARCHIVE SOURCE BCP

47 ARCHIVE POC Environment P3Tradsssss_2000 (16M Rows) Xxx History_2001 (16M Rows) GOOBER Sun E way 7GB Ram IQ Clients BCP Or FROM ASE ODBCJDBCOPENCLODBCJDBCOPENCL SYBASE IQ P3ssssss_2002 VIEWCREATEDVIEWCREATED Ssss _hist_Archive (89M Rows)

48 The Results - Storage

49 The Results - Queries QUERYASESybase IQ 1. SELECT firmNumber, acctNumber, settleDate, tradeDate, sideCode,buySellCode,cusipNbr, secType, quantity, price, principalAmt,grossCommission, tradeProfit, secFee,NetAmount, frontAcctNbr,contraAcctNbr1, tradeInterest,TBASettleDate, entryDate, sourceProgram FROM dbo.P3TradeHistory_2001 WHERE ( sideCode = 'F' ) AND ( cusipNbr = ' L4' ) 49:58.70:40 2. SELECT firmNumber …… FROM P3TradeHistory_2001 WHERE acctNumber = ' ' and sideCode = 'F' 43:43.60:08 3. SELECT firmNumber …… FROM dbo.P3TradeHistory_2002 WHERE ( sideCode = 'F' ) AND -- ( cusipNbr = ' KW' ) (cusipNbr = ' L4') 53:05.90: SELECT firmNumber …… FROM P3TradeHistory_2002 WHERE acctNumber = ' ' and sideCode = 'F' 53:13.60:07

50 Storage Costs Analysis Ref Wachovia Cost Analysis.xlsWachovia Cost Analysis.xls

51 Your Requirements…… The Results Reduce EMC Storage costs = demonstrated Simple and fast implementation = three days for four tables Faster query speeds = all

52 Worlds Largest ISP

53 POC3 Environment SAN EMC SYMM SYBASE IQ V 12.5 ESD GB RAM (800 MB Temp 800MB Main) RED HAT 3.0 Advanced Server ( Elsmp) 14 RAW DEVICES 1-5 TEMP 8GB Each 5-14 DATA 32GB each Data staging Area 100GB HP DL CPU 2.7GHz-2MB cache 4GB RAM

54 Bulk Loading Results 155 GB Per Hour Achieved! A Five-Fold improvement

55 Data Explosion Factors v.s RAW vs. Row-based RDBMS LOAD INPUT DATA: 88 GB 110+GB >IQ Multiplex >Row-Based DBMS FULLY INDEXED ~45 GB DATA 101 GB >RAW Data DATA 88 GB INDEXES 22 GB Aggregates ??? Materialized Views

56 What we demonstrated in POC 3  Great Price/Performance combination >Compression of RAW data (~ 50% of input fully indexed) >Inexpensive/high-performance Hardware >High Performance on a 8-way Linux Blade (Compared to UNIX)  IQ working on HP’s new 32 bit Linux Box Improved performance over 4 way No software issues/downtime  Demonstrated Hyper-threading vs. non-Hyper-threading IQ shows 15% increase with Hyper-Threading ‘on’  Ease of implementation Install and configuration of HW and SW in hours Net POC time a single day Used existing data model and data types with no tweaking  Ease of Administration No P&T on Box, OS or Storage All indexing done during load

57 IQ Write Node HP DL740 8 CPUs 32 GB RAM IQ Reader Nodes 4 CPU Blades from Egenera Storage Currently EMC but not limited

58 Proposed Architecture Source DB Source DB Source DB Source DB Rep Server Rep Connect Possible Target DB Flat Files Sybase IQ Clients Queries Reports Load Process Or ETL Tool


Download ppt "Sybase Update >Joe Shaffner >Regional Technical Manager"

Similar presentations


Ads by Google