Presentation is loading. Please wait.

Presentation is loading. Please wait.

Nicholas Collins Clinical Analytics and Informatics 24 September 2013 A Warehouse and Reporting Architecture for Healthcare using Oracle Technologies [CON3926]

Similar presentations

Presentation on theme: "Nicholas Collins Clinical Analytics and Informatics 24 September 2013 A Warehouse and Reporting Architecture for Healthcare using Oracle Technologies [CON3926]"— Presentation transcript:

1 Nicholas Collins Clinical Analytics and Informatics 24 September 2013 A Warehouse and Reporting Architecture for Healthcare using Oracle Technologies [CON3926]

2 1.About MD Anderson (and me!) 2.The Future of Cancer Treatment and Research 3.Oracle at MD Anderson 4.Our Warehouse and Reporting Architecture 5.Implementation Conclusions Topics 2

3 About MD Anderson 1 3

4 Non-profit cancer hospital and research institution, founded in 1941 as part of The University of Texas System Named after Monroe Dunaway Anderson (a banker and cotton trader, not an MD) Making Cancer History – our mission is to eradicate cancer Consistently ranked as the #1 hospital for cancer care About MD Anderson 4

5 5

6 6 Over 19,000 employees, majority in the Houston area Occupying over 20 buildings in the Texas Medical Center The Texas Medical Center has over 50 member institutions, together over 100,000 employees

7 Began working at MD Anderson while an undergraduate at Rice University (across the street) Currently in the Clinical Analytics and Informatics (CAI) Department, but before that was in HR Information Management (HRIM), working with PeopleSoft and other HR apps While in HRIM, built a custom HR data warehouse and reporting system using a combination of Microsoft and Oracle technologies After hours, a professional stage actor/director in Houston About Me 7

8 The Future of Cancer Treatment and Research 2 8

9 The Time is Now. Together we will end cancer. Target six cancers: Breast/Ovarian, Leukemia (AML/MDS & CLL), Melanoma, Lung, Prostate Clear focus on the concept that the answer to curing cancer lies in both clinical and genomic data MD Anderson Moon Shots Program 9

10 10

11 Its in the Data! 11

12 MD Anderson Moon Shots Platforms 12 Massive Data Analytics – An infrastructure for complex analytics and clinical decision support using integrated patient information, including clinical and research data Big Data – An Information Technology infrastructure/environment that enables centralization, integration and secured access of patient and research data and analytical results

13 Its in the Genes! 13

14 MD Anderson Moon Shots Platforms 14 Clinical Genomics – Clinical gene sequencing infrastructure, including centralized bio-specimen repository and processing Omics – Bioinformatics – A high-throughput infrastructure for generation and standardization of large-scale omic data, including genomics, proteomics and immune profiling Adaptive Learning in Genomic Medicine – A framework for bringing clinical medicine and genomic research together to enable rapid learning to improve patient management using Clinical Genomics, Omics-Bioinformatics and Massive Data Analytics platforms within the Big Data environment

15 Genomics in the News 15

16 Oracle at MD Anderson 3 16

17 Oracle Healthcare Data Warehouse Foundation (HDWF) Oracle Healthcare Analytics Data Integration (OHADI) Oracle TRC (Translational Research Center) Cohort Explorer Oracle TRC Omics Data Bank (ODB) Oracle Health Sciences Products at MD Anderson 17

18 Oracle Database 11gR2 Oracle Exadata (x3) Oracle Business Intelligence (OBIEE) Oracle GoldenGate* Oracle Technology at MD Anderson 18 *Oracle GoldenGate was used to demonstrate replication capabilities in a significant POC, but has not been purchased or put into production. Informatica is commonly used at MD Anderson for data integration; ODI is not currently in use at the institution.

19 Oracle Healthcare Data Warehouse Foundation (HDWF) 19 HDI HDM

20 Oracle Healthcare Analytics Data Integration (OHADI) 20 HDI HDM OHADI Integration code that maps from the interface tables (HDI) to the warehouse tables (HDM) Available as either Informatica or ODI mappings

21 Oracle Cohort Explorer 21 CDM Cohort Explorer

22 Oracle Cohort Explorer 22

23 Oracle Omics Data Bank (ODB) 23

24 24

25 Review of Oracle Health Sciences Products 25 HDI HDM OHADI CDM ODB Cohort Explorer

26 Our Warehouse and Reporting Architecture 4 26

27 FIRE - Federated Institutional Reporting Environment A program level initiative, with many projects and products involved, to provide a unified BI/Reporting solution for all of MD Anderson Managed by the Clinical Analytics and Informatics (CAI) Department, part of Oracle SDP (Strategic Development Parnter) Program The MD Anderson FIRE Program 27

28 FIRE Program Team Structure (Early Proposal) 28

29 FIRE Program Team Structure (Current) 29

30 Custom-built OBIEE Dashboard for orders data, pulling data from HDM, with HDI populated from the GE Centricity source Dimensional model with orders as the core fact Included a smaller pre-release of HR data for staff details and testing the FIRE Architecture Purchased HDWF and OHADI in May 2012, aiming for a fall go-live for our first FIRE release First FIRE Release – Pharmacy Dashboard 30

31 Pharmacy Dashboard (OBIEE) 31

32 Oracle Health Sciences GBU provides products that are a part of an overall solution, the rest is organization specific We needed a way to effectively get data into HDWF and deliver it to custom Data Marts Having a pre-built warehouse model helps with speed of delivery, but not necessarily source system mapping and integration Unifying the Solution 32

33 Bring all the data processing together on a single Oracle instance for performance benefits of local movement and transformation Abstract across all commonalities and patterns to the largest extent possible, avoiding needless one-off solutions, use code generation and automation Ideal candidate for a later forklift to Exadata Architectural Concept 33

34 The FIRE Architecture 34

35 Currently no centralized EHR solution in place, a best-of- breed model with many disparate source systems Data currently brought together for patient-care clinical use in a single UI by a SOA-based custom.NET app called ClinicStation In July 2013, the institution announced its intention to migrate to Epics EHR solution Source Systems at MD Anderson 35

36 The FIRE Architecture 36 SRSI UIUD HDI/HDM

37 Stores replicated data from source systems in the consolidated warehouse environment, provides buffer from sources and their technology, also allows custom indexing and partitioning if needed Replication can be accomplished in a variety of ways, using a bag of dirty tricks to get the data into relational form in Oracle GoldenGate proposed as standard tool to replicate in from relational sources, transparent gateway and Informatica/ODI are other options Repliaction done at table level for consistency and ease of change SR (Staging Replica) Layer 37

38 Pull data directly from the corresponding SR schema (i.e. the SI_CENTRICITY schema pulls data from the SR_CENTRICITY schema) Contains views that match the target HDI tables, one-to-one, same column names and data types Accomplishes selection of the appropriate data, and any necessary pre-transformation In complex cases, can have materialized pre-processing data in tables or materialized views, in practice this meets the 80/20 rule SI (Staging Interface) Layer 38

39 Oracle-defined HDWF interface tables A landing zone schema, can be used as a Persistent Staging Area (PSA) Conceptually, where you place unrefined and unvalidated data to be processed by OHADI before it goes into the HDM (warehouse tables) Tables are designed to be insert-only (source-change dated) HDI Layer 39

40 Oracle-defined HDWF warehouse tables HDM stands for Healtcare Data Model Keys and Referential Integrity (RI) in place for this layer, but RI is disabled by default Conceptually, where all your data is persistently stored, though reloads from HDI are possible Can be configured for effective-dating or only current state HDM Layer 40

41 Similar in concept to the SI Layer, but pulling data from HDM for use in the UD layer Contains views that match the target UD tables, one-to-one, same column names and data types Accomplishes selection of the appropriate data, and any necessary pre-transformation (like SI) In complex cases, can have materialized pre-processing data in tables or materialized views, in practice this meets the 80/20 rule (like SI) UI (User Interface) Layer 41

42 Pull data directly from the corresponding UI schema (i.e. the UD_RX schema pulls data from the views in the UI_RX schema) Contains the user-layer target tables that are used in the dimensional (star schema) models Can be used directly by OBIEE or schemas can be replicated to a separate user database for isolated/off-loaded dashboard processing Data-wise, its the end result of the warehouse pipeline UD (User Data Mart) Layer 42

43 Data Movement 43

44 There was a desire from our integration team to use Informatica for ETL because of experience base on the team, not much PL/SQL or ODI knowledge Architecture proposed use of abstracted code generation via Informatica APIs, jointly used with the push-down optimization option for all non-OHADI internal data movement (i.e. SI to HDI, UI to UD) Data Movement (Planned) 44

45 Our integration team initially indicated that code generation with Informatica (or other tools) could not be done on account of complexity, and that the push-down optimization option was too expensive To demostrate the feasibility, I programmed a PL/SQL-based version of the code generation as proposed in the FIRE Architecture documentation, we used this code in the first release Data Movement (Actual) 45

46 Data Movement Code Generation 46

47 Procedure iv_tv_ip_gen(name_of_sv_view) for SI layer, generates objects for change detection and movement from SR to HDI Procedure iv_uv_dv_ip_gen(name_of_sv_view) for UI layer, generates objects for change detection and movement from HDM to UD All that is needed for generation is the SV view, which conforms to the HDI- based structure, data in certain standard HDI columns determine action A benefit of the generated views is the ability to see what will happen during the next run, without actually running anything PL/SQL Procedures for Code Generation 47

48 Had approximately three months to implement, process was difficult, but in the end everything worked and we went to production with the first FIRE release in November 2012 OHADI was slower than expected but got the job done, Informatica version used, but might be faster with ODI? Integration team wanted a second chance to get code generation going for Informatica, and wanted more Informatica and less SQL and PL/SQL Committee voted to try Informatica alternatives for the next release Results/Next Steps 48

49 In this release, there were four total projects under the FIRE Program: (1) Second Pharmacy Release, (2) Cohort Explorer and ODB (Moon Shot Analytics), (3) Exadata Implementation for Cohort Explorer and ODB, and (4) OBIEE Infrastructure Upgrade Beginning to use Omics Data Bank, data volumes required Exadata License restrictions and no budget yet to put HDWF on Exadata Second FIRE Release – Moon Shot Analytics and Pharmacy Dashboard 49

50 Second Release 50 CDM ODB Cohort Explorer

51 Architectural Changes 51

52 Informatica Code Generation using Java to generate Informatica objects, so not using the PL/SQL code generation with SI and UI views for this release Integration team wanted an instantiated SI Layer and UI Layer for Informatica-based code generation instead of views in the SI and UI Layer Architectural Changes 52

53 Informatica Code Generation with Java 53

54 Successfully implemented all projects within six months, Cohort Explorer and ODB now live, along with the new Pharmacy Dashboard Informatica code generation worked successfully, but perfomance issues surfaced, particularly with the new SI and UI instantiated data The old Pharmacy code (views) was easy to change/update and the integration team did not want to convert old code to the new Informatica- based methodology, hence we have a hybrid model in place currently Some of the Informatica code had to be abandoned close to go-live, replaced by quickly created SQL in views (materialized views) Results 54

55 Had to tinker a little bit with the Exadata install of ODB and CDM, ended up to drop all indexes and implement some materialized views for improved performance, now being incorporated into the tool via our SDP partnership We noticed that Oracle was now using a view-based methodology for the CDM ETL, Informatica mappings use views off of HDM, differs from OHADI, more similar to our original architecture Indexing, partitioning, and SQL tuning were especially necessary in working with our conventional HDWF environment, Exadata can help in the future Aggregation was important to getting OBIEE to perform well with ROLAP More Results 55

56 56 CDM ODB Exadata Implmentation for Cohort Explorer and Omics Data Bank (Moon Shot Analytics)

57 Database Servers Storage Servers Infiniband Switch Database Clients Administrator Exadata Architecture 57

58 Full Rack – 8 DB Servers, 14 Storage Servers Half Rack – 4 DB Servers, 7 Storage Servers Quarter Rack – 2 DB Servers, 3 Storage Servers Eighth Rack – 2 DB Servers*, 3 Storage Servers** Rack Configurations (x3-2) * The eighth racks DB servers each have one processor disabled via software. ** The eighth racks storage servers have half the drives and half the flash cards. 58

59 Exadata x3-2 Capabilities 59

60 Smart Scans (Query Offloading) Storage Indexes Hybrid Columnar Compression Exadata Smart Flash Cache Exadata Database Features These are software-based Exdata-only features. 60

61 Standard Database I/O Exadata I/O select … I/O Request Smart Scan Request Database Server Client Exadata DB Server Exadata Storage Servers Conventional Storage 61

62 CDM/ODB Implementation - Exadata Equipment Purchased Development/Test Environment Production Environment 62

63 Oracle Exadata Readiness Checklist Prepare Network and Power Connections Shipping needs to align with Sun technician arrival SFP Modules – be sure to order if needed Training from Enkitec Exadata Pre-installation Activities 63

64 Photo shown courtesy of Mr. Robert Jeffries, Project Manager 64 MD Anderson CAI War Room Exadata Implementation Team January 2013

65 65

66 Scripts to verify functionality Eighth Rack scripts expected Quarter Rack Most documentation still for x2 equipment, most likely updated by now Exadata Validation Scripts 66

67 Crucial to plan storage allocation in advance Recommended DATA/RECO disk groups: OLTP – 60%/40%, DW 80%/20%, we did a 70%/30% split DB instances share these disk groups per rack Implemented DBFS (Oracle Database File System) for use with the ODB Loaders, omics data files are large, particularly genomic reference data Exadata Storage Allocation 67

68 Performed after storage allocation We installed two separate instances on production rack, four separate instances on dev/test rack All instances are clustered via RAC, ODB and CDM are in the same database instance Instance creation went fairly quickly for us, but would not have been the case for a consolidation project Exadata Instance Creation 68

69 Backup – IBM Tivoli (non-Oracle) Anti-virus Software (non-Oracle) Oracle Enterprise Manager 12c Oracle Platinum Gateway Additional Components 69

70 Oracle Enterprise Manager 12c 70

71 71

72 Single points of contact are ideal (Oracle, Enkitec, Internal Departments) Early engagement project planning important War Room concept very effective Some documentation for new products hard to find (i.e. x3, OEM 12c) Enkitec training was fantastic, but probably should have happened earlier than the delivery, we were on an accelerated implementation schedule Exadata Implementation Lessons Learned 72

73 Assess GoldenGates viability toward MD Anderson Use Cases requiring Heterogeneous Data Replication, Flexible Data Deployment and Continuous Availability Determine ease-of-configuration, deployment, manageability and reliability of GoldenGate as implemented within the use cases Should GoldenGate handle these use cases convincingly, the POC will be considered as successful GoldenGate POC at MD Anderson – July

74 Use Case #1 - Heterogeneous Data Replication from multiple Source Databases, namely, PICIS (CareSuite), OR Manager (Security, Surgery, Interface DBs), and Sybase (RADDATA) to an Oracle 11g R2 Target Database (staging area for HCM Data Warehouse) Use Case #2 - Heterogeneous Data Replication from multiple Source Databases, namely PICIS (CareSuite), OR Manager (Security, Surgery, Interface DBs), and Sybase (RADDATA) to a SQL Server 2008 R2 Target Database (Operations Reporting) Use Case #3 - Flexible Data Deployment topology to merge activity data from multiple Source Tables (Sybase) to a Single SQL Server Target Table (Operations Reporting Database) Use Case #4 - Architecture capable of handling schema differences across platforms (Audit Reporting Columns added at Oracle Target Database) GoldenGate POC Use Cases 74

75 75 GoldenGate POC Architecture

76 GoldenGate POC Results 76

77 Implementation Conclusions 5 77

78 The HDWF, OHADI, Cohort Explorer, and Omics Data Bank products helped us deliver a lot of functionality very quickly, in only one year – our president was very impressed with Cohort Explorer when coupled with ODB, calling it a game changer Having a warehouse model in place helps you avoid a lot of the headaches and time that could be lost to developing your own intermediary models and related enforcement of standards OHADI gives you a good starting point for the HDI to HDM ETL so you can effectively generate your RI relationships and manage exceptions Taking some time to think about our overall architecture and try out some new techniques paid off in the long run, still important to deliver functionality of course Implementation Conclusions 78

79 The HDM tables cover a large swath of clinical concepts, but sometimes things might not fit perfectly, working with Oracle as an SDP has its advantages, as does understanding how to customize the model OHADI is supposed to be the step in the process where data is cleansed and validated, but a lot of customization would be required to do so thoroughly, Oracle will be working to improve this portion of the product, until then you can cleanse pre-HDI if OOTB functionality is not there yet The Informatica version of OHADI had some performance issues on our environments, but I would predict the ODI (Oracle Data Integrator) version would run faster Having a good vocabulary/terminology approach in place will help tremendously with your implementation of code systems in the model, Oracle is beginning to integrate OHADI with HLI Implementation Conclusions 79

80 The Cohort Explorer application aims to deliver the exact functionality that will be needed in cancer research – merging the clinical and genomic data, but the application still needs some more maturity and better use of HDWF structures, we do not want the tail to wag the dog Omics Data Bank helps to centralize a variety of genomics data using set standards from the academic and scientific community, this concept seems to work well in practice The complexity of the queries on ODB and CDM were sometimes challenging from a performance perspective – it is clear that Exadata helps tremendously in this area and we would not have been nearly as successful without it Be sure you have a knowledgeable SME resource for these Oracle technologies to assist you through implementation, they would be difficult to implement on your own Implementation Conclusions 80

81 Get HDWF on Exadata, already ordered two new eighth racks Upgrade Cohort Explorer and ODB to newer versions, when released Focus on architectural refinements/changes for the FIRE Architecture Need to get better code system infrastructure in place, will take work, but worth it in the end – it is unclear how use of Epic/HLI will affect us Beginning NLP pipeline for unstructured data, currently prototyping using IBM ICA platform, but possibly investigate use of Big Data Appliance and/or Hadoop? Semantic technologies? Prepare for Epic as source, hopefully acquire GoldenGate eventually too MD Anderson Future Steps 81

82 Abstraction over commonalities is key in a world driven increasingly by Big Data High-end performance is crucial for BI data delivery, every bit counts Always mind the Mythical Man-Month, the more knowledge and capability a single individual has, the better - streamline your teams for effective agility and delivery, understand the business, beware of design by committee You will never get it right the first time, expect changes and have flexibility to adapt quickly Virtualization, in-memory database objects, and real-time data are gaining momentum in the realm of BI and warehousing, learn to embrace them Collins Axioms – Parting Maxims 82

83 Questions ? 83

Download ppt "Nicholas Collins Clinical Analytics and Informatics 24 September 2013 A Warehouse and Reporting Architecture for Healthcare using Oracle Technologies [CON3926]"

Similar presentations

Ads by Google