CMPE 226 Database Systems October 28 Class Meeting Department of Computer Engineering San Jose State University Fall 2015 Instructor: Ron Mak www.cs.sjsu.edu/~mak

Extract, Transform, and Load (ETL) http://uploads.webflow.com/54e3cc87305a0f0a0665f71f/5537351f5f1065d401cde83c_etl-elt-architecture.jpg

Extract, Transform, and Load (ETL), cont’d http://www.stonebranch.com/common/images/etl-tipping-point.jpg

Extract, Transform, and Load (ETL), cont’d http://superdevelop.com/wp-content/themes/shopperpress/thumbs/ETL.jpeg

Extract, Transform, and Load (ETL), cont’d http://edpflager.com/wp-content/uploads/2014/05/ETL.png

Data Virtualization An approach to data management that allows an application to retrieve and manipulate data without requiring technical details about the data, such as how it is formatted or where it is physically located. An agile data integration approach organizations use to gain more insight from their data, respond faster to ever changing analytics and BI needs and save 50-75% over data replication and consolidation. https://en.wikipedia.org/wiki/Data_virtualization http://www.compositesw.com/data-virtualization/

Data Virtualization, cont’d Differs from traditional ETL. The original data remains in place. A real-time process accesses the data sources. Reduces the risk of data errors. Reduces the workload of extracting, transforming, and loading data that may never be used by an application. Provides applications a “virtual view” of the data. Applications can treat the disparate and heterogeneous data as a single relational database.

Data Virtualization Functionality Abstraction Abstract the technical aspects of stored data, such as location, storage structure, API, access language, and storage technology. Virtualized data access Connect to different data sources and make them accessible from a common logical data access point. Transformation Transform, improve quality, reformat, etc. source data for consumer use. https://en.wikipedia.org/wiki/Data_virtualization

Data Virtualization Functionality, cont’d Data federation Combine result sets from across multiple source systems. Data delivery Publish result sets as views and/or data services executed by client applications or users when requested. https://en.wikipedia.org/wiki/Data_virtualization

Cisco Information Server (CIS) The Cisco Information Server (CIS) forms the core of the Cisco Data Virtualization Platform. http://www.compositesw.com/data-virtualization/

Install the CIS Studio Studio is the client-side application that connects to the CIS server. It’s configured to run only on Windows. Even though it’s a Java application. Download and install Studio. See http://www.cs.sjsu.edu/~mak/CMPE226/CIS Unzip the 2.54 GB zip file, which includes both the CIS server and the client Studio app. Install only Studio. See the Installation Guide.

CIS Team Accounts Each team has a CIS account. Team User name Password Alpha Squad alpha alpha@226 Apollo apollo apollo@226 Blade Runner blade blade@226 Data Crunchers cruncher cruncher@226 Database Predators predator predator@226 Data Nerds nerd nerd@226 Elite elite elite@226 GangOfFour gang gang@226 Oceans3 ocean ocean@226 Sappers sapper sapper@226 Snowlions snow snow@226 Swagg swagg swagg@226 Team Include include include@226

Logging into CIS To log in, start the client-side Studio application. Windows only! Domain: composite Server: 130.65.159.87 Port: 9400

Demo: Cisco Information Server This demo is described in the Getting Started Guide. Query: Why did customer Landmark Systems not receive product Widget 5 on the promised date of February 10, 2003?

Demo: Cisco Information Server, cont’d ViewOrder View the activities of an order. Obtain product status, order date, and customer contact information. ViewSales Obtain information about the purchase department’s interactions with the customer and the supplier. ViewSupplier Use the sales department’s record and noted the lead time for product delivery.

Assignment #8 Each team do the CIS demo described in the Getting Started Guide. Create your sources folder under My Home, not under Shared. You can named the sources folder anything you like. Each team member can have a separate folder. Generate the following screen shots.

Assignment #8, cont’d

Assignment #8, cont’d

Assignment #8, cont’d

Assignment #8, cont’d

Assignment #8, cont’d Tips On page 19 of the Getting Started Guide, after you’ve clicked the Create & Introspect button, you may need to click the Refresh Resource List button. On page 23, the XML file is located on the server in /opt/Composite_Software/CIS_6.2.6/examples

Assignment #8, cont’d Tips, cont’d On page 26, the Create Link and Target is the button circled in red:

Assignment #8, cont’d Email a zip file containing your screen shots to ron.mak@sjsu.edu. Due: Wednesday, Nov. 4 at 11:59 PM. NOTE: We are also creating MySQL accounts for you on the same server. If we accomplish that in the next couple of days, you should upload your database dump file to the server via FTP, recreate your database on the server, and import it into CIS as another data source.

Extra Credit! Convert Studio to run under Bash on Linux or the Mac. Studio is a Java application. Convert the Windows batch file studio.bat (in the CIS bin directory) to a Bash script. Zip all the required directories, jar files, xml files, property files, etc. Unzip and run Studio on Linux or Mac OS X.

Extra Credit! cont’d Create the same screen shots as for Assignment #8. Individual work only! Up to 25 points added to the midterm score of up to 10 students, chosen in the order of midterm scores (lowest scores first). Due Friday, Nov. 13. NOTE: No guarantees that this conversion is even possible!

Project Presentations We lose two days: Nov. 11 Veterans’ Day Nov. 25 Day before Thanksgiving. Originally scheduled for project presentations: Nov. 18 Dec. 2 (last day for this class) We need a new plan for doing project presentations! Two weeks apart!