Forward Data Cache Integration Pattern Ed Allison Regence September 14, 2010

What is a Forward Data Cache? “The Forward Cache integration pattern allows data to be proactively pushed from the backend data sources to a data cache service that is located closer to the presentation tier. Caching data closer to the presentation services improves performance and can allow continuous access when backend sources are offline.” Enterprise Service Bus By David A. Chappell Copyright © 2004

Why build a Forward Data Cache? Availability Performance

How can we populate an FDC? Data Forwarding Enterprise Service Bus Oriented Publish and subscribe Itinerary-based routing ETL Approach

What did we consider technically? Timeliness Tolerable latency and scalability Breadth Data subject areas and detail Depth Amount of history Use Read-only query performance over load performance

What are we using our FDC for? The FDC will be a source of data for enterprise web services that will facilitate HIPAA 5010 transaction processing. For example: Member Search Provider Search 271 Response

How does it fit in our big picture?

Which technologies did we use? Sybase Replication Server Messaging Edition (RSME) Sun Java Composite Application Platform Based on OpenESB Oracle Real Application Cluster (RAC) RDBMS

Why did we make these choices? Sybase RSME Retention of existing Sybase replication use Supports non-Sybase sources Sun JCAPS Our enterprise ESB Oracle RAC Our existing highly available RDBMS

What are we loading into our FDC? Facets Claims System Subscriber / Member Benefits Claims 270/271 Configuration Provider Information Management System Provider

What does our FDC look like?

What are our RSME components?

How do we populate our FDC safely? Source table changes get created as individual messages Use staging tables to hold full source tables (“reference” tables) We need to have the full table available locally for look-ups because the target has been mildly denormalized and update/delete anomalies need to be addressed Carry source primary keys in the target tables

How do we populate our FDC safely? Load the staging and target tables incrementally in a trickle feed, near real-time, manner Load targets in parallel, using autonomous transactions Perform simple data standardizations Rely on source system data quality

What are the rules for population? Source table changes are applied to the target in the order they were made within the context of a single source table and the source primary key, which allows for parallelism in the load Source “reference” table changes apply to 1+N tables: 1 staging table and N denormalized target tables. Source “reference” table inserts, updates and deletes are applied to target tables as mass updates (modifying 0 to many rows in target tables) Source “reference” table deletes cause updates to null values in the target, but only for non-[source]key attributes Denormalized target tables are required to have lookups against staging tables that are denormalized into those targets

What are the rules for population? Deletes in source tables will become “soft deletes” in FDC tables Each denormalized target table will have a fix up process to propagate the reference table changes to the target table The fix up processes are self-scheduling based on their previous runtimes plus a minimum wait time An “Out of band” synchronization checker needs to be used occasionally to verify the FDC is reasonably in sync with its sources

How do we populate our FDC safely?

How to make queries perform? An average 500 msec query response time should be achievable with a tuned RDBMS approach Oracle RAC provides near-linear scalability Tuning options include: Parallelization Partitioning Specialized indexes Virtual columns Workload management SQL Tuning (web services will only generate a limited variations of queries) Materialized views

What alternatives were considered? Object database Considered InterSystems’ Cache In-Memory Data Fabric Oracle Coherence Gemstone Gemfire These were not pursued because The time table for the development of the FDC was not conducive to the introduction of a new data store technology for the FDC A data grid (data fabric) would require extensive engineering to provide a robust, production ready system. A data grid would require significant coding to leverage. Cost due to additional infrastructure required

Are there any questions? Thank you for attending Ed Allison Senior Solutions Engineer Regence ed.allison@regence.com