1. Academic Computing Daniella Meeker, PhD Director, Clinical Research Informatics SC-CTSI Assistant Professor of Preventive Medicine and Pediatrics

2. Why doesn’t health care work like google?

3. Clinical Algorithms vs. Recommender Algorithms GoogleHealth System Platforms for data collection1 per application1000s without interoperability Accumulation of dataContinuous Randomized Trials to improve performance Continuous inexpensive implicit consent Expensive rare, ethical concerns, consent? Number of competing objectives and incentives 2 – user, advertiser5-6, patient, clinician, insurer, pharma, hospital, caregiver Cost of mistakesLow – Learning OpportunitiesHigh Authority for data accessSingleMultiple, including lawyers Distribution of dataMultiple Locations Computation architecturemaster-worker coordinationsilo Analysis execution environment ControlledSerendipitous Incentives for Research Participation HighLow* Distribution and comparison of algorithms softwareliterature

4. Optimization, Evolution & Dissemination of Tools o Optimizing requires an evaluation step o No evaluation platform to analyze next steps o Evaluation requires data

5. Machine Learning/System Science o “Machine” ~ Automation/Efficiency o “Learning” ~ Optimization and improvement

6. What is Data Science? o What is the source of data for data science? –Data about Data (Metadata) o Applications and tools generate metadata about workflow, user experience, and effectiveness –How can we use this to optimize USC use of tools? –Breadcrumbs for collaboration and improvement opportunities come in our application use Time tracking software can do this automatically o Where are incentives for such evaluations in academic computing?

7. Clinical Data Research Networks

8. Common problem in clinical informatics o No platform to compare applications and tools in a head-to-head competition o Tools are developed and not matured after publication o Collaboration costs are even higher than other disciplines…how do we reduce the costs of collaboration? o Need to distribute both master and worker software to collaborate

9. What is global, what is local? o Algorithms are global –Execution environments (software) may be local o Workflow specifications can be global –Workflow execution is local o Data specifications can be global –Data storage can be local or global –Security policies may be local o Regulations for security and privacy are global –Interpretations may be are local

10. Distributing Innovation & Evaluation in Clinical Informatics Tool developers

11. Clinical Informatics at USC Lessons from the mini-DEWARS experiment

12. DEWARS Clinical Research Data Warehouse o Collaboration for enterprise warehouse to be used for biomedical data analytics o Sponsored by SC-CTSI, USC, CHLA o Data harmonization with Los Angeles County, but distributed storage and stewardship o 18 month timeline to first release

13. Mini-DEWARS o Data Set without personal identifiers from CHLA and Keck Electronic Medical Records o I2b2 application “academic standard” for clinical data exploration and cohort identification o Collecting information about data sources and policies around specific test-cases –Test-case #1: the LIBERATE study – handoff back to health system to identify patients for consent and contact –Test-case #2: Los Angeles Data Resource – sharing metadata (patient counts) with UCLA and Cedar’s researchers –Test-case #3: TriNetX – same metadata, different software, industry clients

14. CHLA Keck

15. Lessons Learned about USC from the mini-DEWARS data warehouse experiment o $120K Investment by SC-CTSI (Buchanan, NIH) –“Embedded” and empowered staff at Keck made 6 week process to get from clinical data warehouse to research data warehouse –“Embedded” staff absent at CHLA, no clinical data warehouse, no lines of authority; 6 month process o CHLA and USC centralized research data warehouse –Data are not well understood –Project management styles are very different o The policy infrastructure is more important than technical infrastructure –Funding –Data access –Decision-making authority is fuzzy o Clinical Researchers are innovative, motivated, frustrated o Health system authorities are cautious o Many entrepreneurial aspirations from students o No model yet to ensure benefits are bidirectional to balance risk to health systems with business intelligence benefits back to health system.