Presentation is loading. Please wait.

Presentation is loading. Please wait.

A Warehouse and Reporting Architecture for Healthcare using Oracle Technologies [CON3926] Nicholas Collins Clinical Analytics and Informatics 24 September.

Similar presentations

Presentation on theme: "A Warehouse and Reporting Architecture for Healthcare using Oracle Technologies [CON3926] Nicholas Collins Clinical Analytics and Informatics 24 September."— Presentation transcript:

1 A Warehouse and Reporting Architecture for Healthcare using Oracle Technologies [CON3926]
Nicholas Collins Clinical Analytics and Informatics 24 September 2013

2 Topics About MD Anderson (and me!)
The Future of Cancer Treatment and Research Oracle at MD Anderson Our Warehouse and Reporting Architecture Implementation Conclusions 2

3 1 About MD Anderson 3

4 About MD Anderson Non-profit cancer hospital and research institution, founded in as part of The University of Texas System Named after Monroe Dunaway Anderson (a banker and cotton trader, not an MD) “Making Cancer History” – our mission is to eradicate cancer Consistently ranked as the #1 hospital for cancer care 4

5 About MD Anderson 5

6 About MD Anderson Over 19,000 employees, majority in the Houston area
Occupying over 20 buildings in the Texas Medical Center The Texas Medical Center has over 50 member institutions, together over 100,000 employees 6

7 About Me Began working at MD Anderson while an undergraduate at Rice University (across the street) Currently in the Clinical Analytics and Informatics (CAI) Department, but before that was in HR Information Management (HRIM), working with PeopleSoft and other HR apps While in HRIM, built a custom HR data warehouse and reporting system using a combination of Microsoft and Oracle technologies After hours, a professional stage actor/director in Houston 7

8 The Future of Cancer Treatment and Research
2 The Future of Cancer Treatment and Research 8

9 MD Anderson Moon Shots Program
“The Time is Now. Together we will end cancer.” Target six cancers: Breast/Ovarian, Leukemia (AML/MDS & CLL), Melanoma, Lung, Prostate Clear focus on the concept that the answer to curing cancer lies in both clinical and genomic data 9

10 MD Anderson Moon Shots Program

11 It’s in the Data! 11

12 MD Anderson Moon Shots Platforms
Massive Data Analytics – An infrastructure for complex analytics and clinical decision support using integrated patient information, including clinical and research data Big Data – An Information Technology infrastructure/environment that enables centralization, integration and secured access of patient and research data and analytical results 12

13 It’s in the Genes! 13

14 MD Anderson Moon Shots Platforms
Clinical Genomics – Clinical gene sequencing infrastructure, including centralized bio-specimen repository and processing Omics – Bioinformatics – A high-throughput infrastructure for generation and standardization of large-scale “omic” data, including genomics, proteomics and immune profiling Adaptive Learning in Genomic Medicine – A framework for bringing clinical medicine and genomic research together to enable rapid learning to improve patient management using Clinical Genomics, Omics-Bioinformatics and Massive Data Analytics platforms within the Big Data environment 14

15 Genomics in the News 15

16 3 Oracle at MD Anderson 16

17 Oracle Health Sciences Products at MD Anderson
Oracle Healthcare Data Warehouse Foundation (HDWF) Oracle Healthcare Analytics Data Integration (OHADI) Oracle TRC (Translational Research Center) Cohort Explorer Oracle TRC Omics Data Bank (ODB) 17

18 Oracle Technology at MD Anderson
Oracle Database 11gR2 Oracle Exadata (x3) Oracle Business Intelligence (OBIEE) Oracle GoldenGate* *Oracle GoldenGate was used to demonstrate replication capabilities in a significant POC, but has not been purchased or put into production. Informatica is commonly used at MD Anderson for data integration; ODI is not currently in use at the institution. 18

19 Oracle Healthcare Data Warehouse Foundation (HDWF)

20 Oracle Healthcare Analytics Data Integration (OHADI)
HDM HDI Integration code that maps from the interface tables (HDI) to the warehouse tables (HDM) Available as either Informatica or ODI mappings 20

21 Oracle Cohort Explorer
CDM 21

22 Oracle Cohort Explorer

23 Oracle Omics Data Bank (ODB)

24 24

25 Review of Oracle Health Sciences Products
Cohort Explorer HDI HDM CDM OHADI ODB 25

26 Our Warehouse and Reporting Architecture
4 Our Warehouse and Reporting Architecture 26

27 The MD Anderson FIRE Program
FIRE - Federated Institutional Reporting Environment A program level initiative, with many projects and products involved, to provide a unified BI/Reporting solution for all of MD Anderson Managed by the Clinical Analytics and Informatics (CAI) Department, part of Oracle SDP (Strategic Development Parnter) Program 27

28 FIRE Program Team Structure (Early Proposal)

29 FIRE Program Team Structure (Current)

30 First FIRE Release – Pharmacy Dashboard
Custom-built OBIEE Dashboard for orders data, pulling data from HDM, with HDI populated from the GE Centricity source Dimensional model with orders as the core fact Included a smaller pre-release of HR data for staff details and testing the FIRE Architecture Purchased HDWF and OHADI in May 2012, aiming for a fall go-live for our first FIRE release 30

31 Pharmacy Dashboard (OBIEE)

32 Unifying the Solution Oracle Health Sciences GBU provides products that are a part of an overall solution, the rest is organization specific We needed a way to effectively get data into HDWF and deliver it to custom Data Marts Having a pre-built warehouse model helps with speed of delivery, but not necessarily source system mapping and integration 32

33 Architectural Concept
Bring all the data processing together on a single Oracle instance for performance benefits of local movement and transformation Abstract across all commonalities and patterns to the largest extent possible, avoiding needless one-off solutions, use code generation and automation Ideal candidate for a later “forklift” to Exadata 33

34 The FIRE Architecture 34

35 Source Systems at MD Anderson
Currently no centralized EHR solution in place, a best-of- breed model with many disparate source systems Data currently brought together for patient-care clinical use in a single UI by a SOA-based custom .NET app called ClinicStation In July 2013, the institution announced its intention to migrate to Epic’s EHR solution 35

36 The FIRE Architecture HDI/HDM SR SI UI UD 36

37 SR (Staging Replica) Layer
Stores replicated data from source systems in the consolidated warehouse environment, provides buffer from sources and their technology, also allows custom indexing and partitioning if needed Replication can be accomplished in a variety of ways, using a “bag of dirty tricks” to get the data into relational form in Oracle GoldenGate proposed as standard tool to replicate in from relational sources, transparent gateway and Informatica/ODI are other options Repliaction done at table level for consistency and ease of change 37

38 SI (Staging Interface) Layer
Pull data directly from the corresponding SR schema (i.e. the SI_CENTRICITY schema pulls data from the SR_CENTRICITY schema) Contains views that match the target HDI tables, one-to-one, same column names and data types Accomplishes selection of the appropriate data, and any necessary pre-transformation In complex cases, can have materialized pre-processing data in tables or materialized views, in practice this meets the 80/20 rule 38

39 HDI Layer Oracle-defined HDWF interface tables
A “landing zone” schema, can be used as a Persistent Staging Area (PSA) Conceptually, where you place unrefined and unvalidated data to be processed by OHADI before it goes into the HDM (warehouse tables) Tables are designed to be insert-only (source-change dated) 39

40 HDM Layer Oracle-defined HDWF warehouse tables
HDM stands for Healtcare Data Model Keys and Referential Integrity (RI) in place for this layer, but RI is disabled by default Conceptually, where all your data is persistently stored, though reloads from HDI are possible Can be configured for effective-dating or only current state 40

41 UI (User Interface) Layer
Similar in concept to the SI Layer, but pulling data from HDM for use in the UD layer Contains views that match the target UD tables, one-to-one, same column names and data types Accomplishes selection of the appropriate data, and any necessary pre-transformation (like SI) In complex cases, can have materialized pre-processing data in tables or materialized views, in practice this meets the 80/20 rule (like SI) 41

42 UD (User Data Mart) Layer
Pull data directly from the corresponding UI schema (i.e. the UD_RX schema pulls data from the views in the UI_RX schema) Contains the user-layer target tables that are used in the dimensional (star schema) models Can be used directly by OBIEE or schemas can be replicated to a separate user database for isolated/off-loaded dashboard processing Data-wise, it’s the end result of the warehouse pipeline 42

43 Data Movement 43

44 Data Movement (Planned)
There was a desire from our integration team to use Informatica for ETL because of experience base on the team, not much PL/SQL or ODI knowledge Architecture proposed use of abstracted code generation via Informatica APIs, jointly used with the push-down optimization option for all non-OHADI internal data movement (i.e. SI to HDI, UI to UD) 44

45 Data Movement (Actual)
Our integration team initially indicated that code generation with Informatica (or other tools) could not be done on account of complexity, and that the push-down optimization option was too expensive To demostrate the feasibility, I programmed a PL/SQL-based version of the code generation as proposed in the FIRE Architecture documentation, we used this code in the first release 45

46 Data Movement Code Generation

47 PL/SQL Procedures for Code Generation
Procedure iv_tv_ip_gen(name_of_sv_view) for SI layer, generates objects for change detection and movement from SR to HDI Procedure iv_uv_dv_ip_gen(name_of_sv_view) for UI layer, generates objects for change detection and movement from HDM to UD All that is needed for generation is the SV view, which conforms to the HDI- based structure, data in certain standard HDI columns determine action A benefit of the generated views is the ability to see what will happen during the next run, without actually running anything 47

48 Results/Next Steps Had approximately three months to implement, process was difficult, but in the end everything worked and we went to production with the first FIRE release in November 2012 OHADI was slower than expected but got the job done, Informatica version used, but might be faster with ODI? Integration team wanted a second chance to get code generation going for Informatica, and wanted more Informatica and less SQL and PL/SQL Committee voted to try Informatica alternatives for the next release 48

49 Second FIRE Release – Moon Shot Analytics and Pharmacy Dashboard
In this release, there were four total projects under the FIRE Program: (1) Second Pharmacy Release, (2) Cohort Explorer and ODB (Moon Shot Analytics), (3) Exadata Implementation for Cohort Explorer and ODB, and (4) OBIEE Infrastructure Upgrade Beginning to use Omics Data Bank, data volumes required Exadata License restrictions and no budget yet to put HDWF on Exadata 49

50 Second Release Cohort Explorer CDM ODB 50

51 Architectural Changes

52 Architectural Changes
Informatica Code Generation using Java to generate Informatica objects, so not using the PL/SQL code generation with SI and UI views for this release Integration team wanted an instantiated SI Layer and UI Layer for Informatica-based code generation instead of views in the SI and UI Layer 52

53 Informatica Code Generation with Java

54 Results Successfully implemented all projects within six months, Cohort Explorer and ODB now live, along with the new Pharmacy Dashboard Informatica code generation worked successfully, but perfomance issues surfaced, particularly with the new SI and UI instantiated data The old Pharmacy code (views) was easy to change/update and the integration team did not want to convert old code to the new Informatica- based methodology, hence we have a hybrid model in place currently Some of the Informatica code had to be abandoned close to go-live, replaced by quickly created SQL in views (materialized views) 54

55 More Results Had to tinker a little bit with the Exadata install of ODB and CDM, ended up to drop all indexes and implement some materialized views for improved performance, now being incorporated into the tool via our SDP partnership We noticed that Oracle was now using a view-based methodology for the CDM ETL, Informatica mappings use views off of HDM, differs from OHADI, more similar to our original architecture Indexing, partitioning, and SQL tuning were especially necessary in working with our conventional HDWF environment, Exadata can help in the future Aggregation was important to getting OBIEE to perform well with ROLAP 55

56 Exadata Implmentation for Cohort Explorer and Omics Data Bank
(Moon Shot Analytics) CDM ODB 56

57 Exadata Architecture Database Clients Administrator Database Servers
Infiniband Switch Storage Servers 57

58 Rack Configurations (x3-2)
Full Rack – 8 DB Servers, 14 Storage Servers Half Rack – 4 DB Servers, 7 Storage Servers Quarter Rack – 2 DB Servers, 3 Storage Servers Eighth Rack – 2 DB Servers*, 3 Storage Servers** * The eighth rack’s DB servers each have one processor disabled via software. ** The eighth rack’s storage servers have half the drives and half the flash cards. 58

59 Exadata x3-2 Capabilities

60 Exadata Database Features
Smart Scans (Query Offloading) Storage Indexes Hybrid Columnar Compression Exadata Smart Flash Cache These are software-based “Exdata-only” features. 60

61 Standard Database I/O Exadata I/O
Conventional Storage select … I/O Request Client Database Server Exadata I/O Exadata Storage Servers Smart Scan Request select … Client Exadata DB Server 61

62 CDM/ODB Implementation - Exadata Equipment Purchased
Development/Test Environment Production Environment 62

63 Exadata Pre-installation Activities
Oracle Exadata Readiness Checklist Prepare Network and Power Connections Shipping needs to align with Sun technician arrival SFP Modules – be sure to order if needed Training from Enkitec 63

64 MD Anderson CAI “War Room” Exadata Implementation Team
January 2013 64 Photo shown courtesy of Mr. Robert Jeffries, Project Manager

65 65

66 Exadata Validation Scripts
Scripts to verify functionality Eighth Rack scripts expected Quarter Rack Most documentation still for x2 equipment, most likely updated by now 66

67 Exadata Storage Allocation
Crucial to plan storage allocation in advance Recommended DATA/RECO disk groups: OLTP – 60%/40%, DW 80%/20%, we did a 70%/30% split DB instances share these disk groups per rack Implemented DBFS (Oracle Database File System) for use with the ODB Loaders, omics data files are large, particularly genomic reference data 67

68 Exadata Instance Creation
Performed after storage allocation We installed two separate instances on production rack, four separate instances on dev/test rack All instances are clustered via RAC, ODB and CDM are in the same database instance Instance creation went fairly quickly for us, but would not have been the case for a consolidation project 68

69 Additional Components
Backup – IBM Tivoli (non-Oracle) Anti-virus Software (non-Oracle) Oracle Enterprise Manager 12c Oracle Platinum Gateway 69

70 Oracle Enterprise Manager 12c

71 71

72 Exadata Implementation Lessons Learned
Single points of contact are ideal (Oracle, Enkitec, Internal Departments) Early engagement project planning important “War Room” concept very effective Some documentation for new products hard to find (i.e. x3, OEM 12c) Enkitec training was fantastic, but probably should have happened earlier than the delivery, we were on an accelerated implementation schedule 72

73 GoldenGate POC at MD Anderson – July 2012
Assess GoldenGate’s viability toward MD Anderson Use Cases requiring Heterogeneous Data Replication, Flexible Data Deployment and Continuous Availability Determine ease-of-configuration, deployment, manageability and reliability of GoldenGate as implemented within the use cases “Should GoldenGate handle these use cases convincingly, the POC will be considered as successful” 73

74 GoldenGate POC Use Cases
Use Case #1 - Heterogeneous Data Replication from multiple Source Databases, namely, PICIS (CareSuite), OR Manager (Security, Surgery, Interface DBs), and Sybase (RADDATA) to an Oracle 11g R2 Target Database (staging area for HCM Data Warehouse) Use Case #2 - Heterogeneous Data Replication from multiple Source Databases, namely PICIS (CareSuite), OR Manager (Security, Surgery, Interface DBs), and Sybase (RADDATA) to a SQL Server 2008 R2 Target Database (Operations Reporting) Use Case #3 - Flexible Data Deployment topology to merge activity data from multiple Source Tables (Sybase) to a Single SQL Server Target Table (Operations Reporting Database) Use Case #4 - Architecture capable of handling schema differences across platforms (Audit Reporting Columns added at Oracle Target Database) 74

75 GoldenGate POC Architecture

76 GoldenGate POC Results

77 Implementation Conclusions
5 Implementation Conclusions 77

78 Implementation Conclusions
The HDWF, OHADI, Cohort Explorer, and Omics Data Bank products helped us deliver a lot of functionality very quickly, in only one year – our president was very impressed with Cohort Explorer when coupled with ODB, calling it a “game changer” Having a warehouse model in place helps you avoid a lot of the headaches and time that could be lost to developing your own intermediary models and related enforcement of standards OHADI gives you a good starting point for the HDI to HDM ETL so you can effectively generate your RI relationships and manage exceptions Taking some time to think about our overall architecture and try out some new techniques paid off in the long run, still important to deliver functionality of course 78

79 Implementation Conclusions
The HDM tables cover a large swath of clinical concepts, but sometimes things might not fit perfectly, working with Oracle as an SDP has its advantages, as does understanding how to customize the model OHADI is supposed to be the step in the process where data is cleansed and validated, but a lot of customization would be required to do so thoroughly, Oracle will be working to improve this portion of the product, until then you can cleanse pre-HDI if OOTB functionality is not there yet The Informatica version of OHADI had some performance issues on our environments, but I would predict the ODI (Oracle Data Integrator) version would run faster Having a good vocabulary/terminology approach in place will help tremendously with your implementation of code systems in the model, Oracle is beginning to integrate OHADI with HLI 79

80 Implementation Conclusions
The Cohort Explorer application aims to deliver the exact functionality that will be needed in cancer research – merging the clinical and genomic data, but the application still needs some more maturity and better use of HDWF structures, we do not want the tail to wag the dog Omics Data Bank helps to centralize a variety of genomics data using set standards from the academic and scientific community, this concept seems to work well in practice The complexity of the queries on ODB and CDM were sometimes challenging from a performance perspective – it is clear that Exadata helps tremendously in this area and we would not have been nearly as successful without it Be sure you have a knowledgeable SME resource for these Oracle technologies to assist you through implementation, they would be difficult to implement on your own 80

81 MD Anderson Future Steps
Get HDWF on Exadata, already ordered two new eighth racks Upgrade Cohort Explorer and ODB to newer versions, when released Focus on architectural refinements/changes for the FIRE Architecture Need to get better code system infrastructure in place, will take work, but worth it in the end – it is unclear how use of Epic/HLI will affect us Beginning NLP pipeline for unstructured data, currently prototyping using IBM ICA platform, but possibly investigate use of Big Data Appliance and/or Hadoop? Semantic technologies? Prepare for Epic as source, hopefully acquire GoldenGate eventually too 81

82 Collins Axioms – Parting Maxims
Abstraction over commonalities is key in a world driven increasingly by “Big Data” High-end performance is crucial for BI data delivery, every bit counts Always mind the “Mythical Man-Month,” the more knowledge and capability a single individual has, the better - streamline your teams for effective agility and delivery, understand the business, beware of “design by committee” You will never get it right the first time, expect changes and have flexibility to adapt quickly Virtualization, in-memory database objects, and real-time data are gaining momentum in the realm of BI and warehousing, learn to embrace them 82

83 Questions? 83

Download ppt "A Warehouse and Reporting Architecture for Healthcare using Oracle Technologies [CON3926] Nicholas Collins Clinical Analytics and Informatics 24 September."

Similar presentations

Ads by Google