How are data moved from operational systems into a data warehouse? 3 Step 1. Snapshots are extracted from operational systems. Step 2. Extracted files are reformatted and cleaned. Step 3. Pre-processed files are loaded into staging tables and metadata are loaded into lookup tables in an Oracle relational database. Step 4. Data in the staging tables are migrated to normalized tables. Step 5. Summary tables and other high performance query structures are created from the normalized tables and lookup tables. Step 6. Semester-based fact and dimension tables are created from the normalized tables and lookup tables. Step 7. Longitudinal fact tables are created from the semester- based fact and dimension tables.
Steps 1–3 Why are snapshots used to populate the IRDB? 4 SIMS Show- Registration File SPSS Clean Show-Reg File SQL*Loader SHOW_FILE Extraction COBOL TransformationLoad Academic Program Inventory NYSED_PPROGRAM_LOOKUP Database LinkDatabase View SKAT Performance File Graduation File Post Graduate Surveys Skills Tests Results NCS Pearsons COBOL PERF_FILE GRAD_FILE SKAT_FILE_02 VTEA_SURVEY_FILE_02 Clean Performance File Clean Graduation File SPSS SQL*Loader SPSS Survey data with SSN’s SQL*Loader
5 CUNY IRDB Data Flow Diagram CAS (freshman admissions) Special Reports Standardized Files Joins from Multiple Tables across Multiple Terms Group by Selected Columns (SQL) Oracle Discoverer Crosstabs Ad-Hoc Queries Migrate Data into Oracle9i Environment (SQL*Loader) Normalize Data (PL/SQL) Oracle Forms CUNY Data Book on Institutional Research Web Site Extract Files Oracle Discoverer Tables ASTA (transfer admissions) SHOW (enrollment) SKAT (skills tests) PERF (grades) GRAD (degrees) NCES (job survey) SFA (financial aid) Clearinghouse (transfers to non-CUNY colleges) Reformat and Clean Input Files (SPSS) Create Fact and Dimension Tables (SQL) Migrate Data into Oracle 9i Environment (SQL*Loader) Type or Cut and Paste SPSS for Windows Crystal Reports and Oracle Portal Institutional Researchers University Administrators Public Users Oracle Discoverer Crosstabs Staging Tables Code Descriptions from File Layouts PC Files Operational Data Store (normalized student-level data) Lookup Tables (metadata) Flash Enrollment Summary Tables (denormalized aggregate-level data) Data Warehhouse (denormalized student-level data) Longitudinal Cohorts (denormalized student-level data) Ad-Hoc Queries Spread sheets
What are fact and dimension tables and how are they related? 7 A fact table is composed of numerical measures of business performance. Examples of facts would be headcount, FTE’s, and cumulative credits earned. Dimension tables contain items that describe or categorize the items in the fact table. Examples of dimensions would be gender, full-time/part-time status, and college of attendance. The fact table also contains foreign keys that can be used to join it with the primary keys of the dimension tables. For example, “Student ID”, “Term Enrolled Date”, and “College ID” are used to join the table “History Facts” with the table “History Major 1 Dim”. A central fact table with multiple dimension tables radiating out from it is called a star schema.
What are the advantages of using a star schema? 8 Creates a database design that improves performance. Parallels, in the database design, how the end users usually think and use the data. Provides versatile and robust ad-hoc query capabilities. Provides an extensible design which supports changing business requirements. Can be used with point-and-click tools such as Oracle Discoverer 9iAs.
The Joins between the Fact Table “History Facts” and its Dimension Tables Are Defined by an OIRA Administrator in the Discoverer End-User Layer 10
How is a campus limited to viewing only the data of its own students? 11 IR.USERID_LOOKUP # userid # college_id # table_name # table_grant IR.HISTORY_FACTS # student_id # term_enrolled_date # college_id IR.SEC_COLLEGE_07_MV # sec_student_id IRASI Institutional Research Staten Island IRASI.HISTORY_FACTS # student_id # term_enrolled_date # college_id
Users Select “Items” from a “Folder” with a Mouse Rather than Writing and Executing SQL Code 12
Discoverer 13 IRDB End-User Query Tool Currently accessed via Citrix Requires user id/password – domain log in (managed by CIS) Discoverer (account required – managed by OIRA) Set of Business Areas (linked fact and dimension tables) History Facts – Historical Enrollment Records Degree Facts - Historical Degree Records (through most recent complete academic year) Cohort Facts – Integration of Enrollment and Degree data in a longitudinal structure for tracking cohorts over time Special Business Area - mostly stand-alone tables for specific analyses (e.g., PMP)
Users Arrange the Items as the Page-Breaks, Columns, and Rows for a Desired Report 14
Accessing the IRDB Through Discoverer 15 Navigate your web browser to https://ez.cuny.eduhttps://ez.cuny.edu Log in with your LAN user id and password Click on the Discoverer icon in the list of available applications via Citrix Install Java code as prompted upon first use of a given computer (you may need an IT technician to install programs on your computer) After Java installation, you will be prompted to log in to Discoverer (user id and initial password established by OIRA) Documentation available
Creating a New Workbook as a Crosstabs Report with Discoverer 9iAS 16
The Derived Fact “Headcount” Reflects the Business Rules for Excluding Some Students from Official Enrollment Statistics 17