Making Workflows Work  Prof. Yike Guo  Dept. of Computing  Imperial College London  InforSense Limited 

Proprietary and Confidential DiscoveryNet Project  Funding  One of the Eight UK National e-Science Projects (£2.2 M)  Sept 2001 – March 2005  Partners  Achievements  Constructing the World’s First Infrastructure for Building Analytical Services by Scientists  For the First time Discovery Net Realises the Dynamic Construction of Compositional Services on GRID for Real Time Knowledge Discovery and Decision Making  Outputs  Software Research: DNet platform commercialized by InforSense Ltd (>100 customers)  Total user numbers > 2000  Applications Research: Application out  puts in sensor technology commercialized by deltaDot Ltd  Number papers published: 10 Journal Papers, 30 Conference Papers  8 PhD completed and 50 Master students  Ranked OUTSTANDING at the project final review

Proprietary and Confidential InforSense Introduction 100+ customers (70% Fortune 200 companies)  rd fastest growing company in UK (Sunday Times Tech Track)  th fastest growing venture-based company in UK (Financial Times)  Global footprint with offices in London (HQ and R/D), Boston (USA HQ) and Shanghai (Asia HQ and Development base)  Global sales with 70% outside of Europe  7 years of delivering products and services to pharmaceutical and Financial industries  Spin out from Imperial College London Invented “ Distributed Data Mining ” ‘98 First Enterprise Deployment Embedding Analytics Technology 3rd fast growing company in UK ‘01 ‘ ,03 IEEE Super- computing Award – Grid based analytics InforSense Formed Introduced KDE Analytics Platform Discovery Net Project Embedding Analytics in Major Enterprise Systems ‘06 ‘00 Innovation in Embedding Analytics

Proprietary and Confidential CAMBRIA BIOSCIENCES Those who are using our workflow

Proprietary and Confidential Excel EMR Databases Oracle Pre-processing Oracle Pre-processing 3rd party Analytics 3rd party Analytics Web services Biomedical Informatics tools Biomedical Informatics tools Multiple data sources Multiple data sources Interactive Knowledge Discovery Interactive Solution Building Rapid Application Deployment Portal / Dashboard Application InforSense Workflow Methodology Files Automation & Scheduling Data Applications Components InforSense Analytics InforSense Analytics Integrative Analytics Workflow Environment Delivery to End User Dynamic Data & App Integration Business Process AdministratorClinicianDisease Biologist

Proprietary and Confidential What is InforSense WF System Designed for ?  InforSense workflow system is not an application but a framework to build and deliver applications directly to scientist/business user: Chem-Studio ADMET Browser

Proprietary and Confidential Pipelining Web Service Orchestration ETL Enterprise Service Bus Data/Text Mining Business Process Managenment Simulation & Modelling InforSense Generic Workflow Engine

Proprietary and Confidential Experience of 7 years in WF business  Building workflow is easy !  However,  Building a USABLE workflow is not easy  Building a REUSABLE workflow is hard  Building a REUSABLE workflow applications is very hard  Building a REUSABLE workflow application for EVERYONE is very very hard  Building a function is easy, building an application is hard, it is even harder if we enable a non-IT person to build a good reliable application for other people to use everyday!

Proprietary and Confidential InforSense Workflow System Development Workflow Execution Reliable Enterprise Wide Execution Workflow Management Collaborative Knowledge Management Workflow Deployment: Building Reusable WF Applications Workflow Warehousing Resource Mapping Service Abstraction Workflow Authoring Composing services Condor-G Native MPI OGSA-service Web Service Unicore Oralce 10g Web Wrapper Sun Grid Engine Workflow Embedding Pervasive WF applications

Proprietary and Confidential Three Tiers of Workflow Framework Building Layer Application Layer Embedding Layer Analytical Workflow Development Rapid Application Development Service Orchestration Business Rules Embed in Other Applications Analytic Service Encapsulation Publish Services for Display BPEL

InforSense Workflow Building: Not about another graph notation but about how to build a meaningful graph

Proprietary and Confidential Current model of workflow authoring/execution  No help provided to user (authoring/execution)  Model is based on expert user who know about services  Model requires user to be trained in a workflow language/system  Interoperability between workflow systems is only at run-time

Proprietary and Confidential The key the success : End User Oriented Workflow Construction  Build semi-automatic tools that advise/assist user in wf authoring  Make use of previous knowledge about developing workflows  Explicit/Expert knowledge  Implicit knowledge in previous workflows The aim is to help user, not replace him

Proprietary and Confidential Guided Workflow Construction  User is presented by high-level descriptions of predefined task steps  User is guided iteratively in instantiating the task descriptions using workflow templates  User can retrieve workflows and workflow templates from repository  Approach supports using workflows from multiple systems using existing run-time interoperability mechanisms

Proprietary and Confidential Workflow Advisor: InforSense Customer Hubs

Proprietary and Confidential Extended infrastructure: Workflow warehousing and mining  Workflow Advisor  Initial implementations of prototype for bio applications  Workflow Assistant  Abstract component initial prototypes  Workflow Mining  Repository of workflows from Southampton  Workflow Annotations  independent from workflow language  Warehouse  Search and execute web services/Grid services and workflows  Syntactic and semantic search

Proprietary and Confidential Extended infrastructure: Workflow warehouse/registry

InforSense Embedding and Deployment Workflow output is not a data, but an application/service

Proprietary and Confidential InforSense KDE Deployment Strategies Deploy workflows to InforSense portal  Deployment features: multi-page, service chain, layout editor  Multi-stage applications: group workflows into stages  Component based deployment  Portlet based deployment  Portlet component: JSR 168 compatible portlet components Business process workflow  Based on control flow orchestrated workflows and role based deployment

Proprietary and Confidential Web-based Deployment Portal Container allows users to build dashboards Each Workflow generate data for a dashboard component Workflow results viewed in simple charts - can be linked to other pages

Proprietary and Confidential Deployment Features (2) Define multiple pages Move to next page

Proprietary and Confidential Chip QCNormaliseAnalyseInterpret Design Experiment Design Study groups for transcriptomics portal Gene Expression Profiling Pre-process and Analyse the results of a gene expression analysis to compare control vs. test populations Splice Variance Analysis Pre-process and Analyse the results of an Exon Chip to find differences in splice variance between control vs. test populations Results Normalisation services RMA (recommended) LiWong ETC Next Steps Submit to Report> Example Application Analytical stage Workflow configured to group according to stage Portal look and feel can be customized by style sheet

Proprietary and Confidential Chip QCNormaliseAnalyseInterpret Design Experiment Design Study groups for transcriptomics portal Gene Expression Profiling Pre-process and Analyse the results of a gene expression analysis to compare control vs. test populations Splice Variance Analysis Pre-process and Analyse the results of an Exon Chip to find differences in splice variance between control vs. test populations Results Analysis services Volcano Plot (recommended) PCA Dendrogram Next Steps Submit to Report> Example Application

Proprietary and Confidential Chip QCNormaliseAnalyseInterpret Design Experiment Design Study groups for transcriptomics portal Gene Expression Profiling Pre-process and Analyse the results of a gene expression analysis to compare control vs. test populations Splice Variance Analysis Pre-process and Analyse the results of an Exon Chip to find differences in splice variance between control vs. test populations Results Next Steps Submit to Report> Save Result to Report Analysis services Select Transcripts Filter Data Example Application

Proprietary and Confidential Chip QCNormaliseAnalyseInterpret Design Experiment Design Study groups for transcriptomics portal Gene Expression Profiling Pre-process and Analyse the results of a gene expression analysis to compare control vs. test populations Splice Variance Analysis Pre-process and Analyse the results of an Exon Chip to find differences in splice variance between control vs. test populations Results Next Steps Submit to Report> Save Selected Items to Report Interpretation services Send Data to Ingenuity Send Data to Gene Go Send Data to Text Analysis Example Application

Proprietary and Confidential Chip QCNormaliseAnalyseInterpret Design Experiment Design Study groups for transcriptomics portal Gene Expression Profiling Pre-process and Analyse the results of a gene expression analysis to compare control vs. test populations Splice Variance Analysis Pre-process and Analyse the results of an Exon Chip to find differences in splice variance between control vs. test populations Results Next Steps Submit to Report> Interpretation services Send Data to Gene Go Text Analysis Save to Report Example Application

Proprietary and Confidential Chip QCNormaliseAnalyseInterpret Design Experiment Design Study groups for transcriptomics portal Gene Expression Profiling Pre-process and Analyse the results of a gene expression analysis to compare control vs. test populations Splice Variance Analysis Pre-process and Analyse the results of an Exon Chip to find differences in splice variance between control vs. test populations Results Next Steps Submit to Report> Interpretation services Send Data to Gene Go Text Analysis Select Subset for Text Analysis Example Application

Proprietary and Confidential Business Process Management Development  A Business Process Management (BPM) describes the orchestration of different tasks to complete a specific business objective  Business Processes need to orchestrate  Automated Tasks  User Tasks  Exception Handling  Running Tasks in parallel  Synchronisation of parallel tasks Business Process Workflow (1)

Proprietary and Confidential InforSense Control Flow  InforSense Control Flow for Orchestrating Workflows for Business Process Run Task Handle Exceptions Initiate Parallel Tasks Synchronize Parallel Tasks Apply Rules Business Process Workflow (2)

Proprietary and Confidential Orchestra business analytics by control flow Workflow AWorkflow B Workflow C Sub-process 1 Sub-process 2 Control Flow Represents a Business Process Deploy to Portal Application Building Blocks services Process Building Blocks definition of linkage/control and user interactions Business Process Workflow (3)

Proprietary and Confidential Workflow interoperability Workflows and business processes (BPEL)

Proprietary and Confidential Embedding Workflow Analytics into Applications Process View Lifetime Value Service Risk Service Churn Service Embeddable Analytic Applications Analytical Workflows Model Repository Business Rules and Model Deploy New Actions customer data Predictive scores Risk data Risk Evaluation Acceptable Risk? Yes No Get Value Score Normal Service Get Churn Score Risk Assessment Upgrade offer KVM

Proprietary and Confidential Integrating Analytics with Business Rules: Adaptive Business Process Enterprise Services Bus Business Process Business Portal Business operational data Analytics to drive adaptive processes Rule engine Rule Engine

Proprietary and Confidential Embedding with Applications InforSense Tools as one item in Windows based application system

Proprietary and Confidential

“ One of the biggest barriers to achieving productivity and responsiveness is IT – it has become a bottleneck. Another barrier to achieving the goal is the lack of intelligence that drives most IT applications. They are just operating as a rapid functional replacement, and failing to exploit the data which is being generated within other elements of the IT infrastructure. A product that could meet that challenge and enable business to generate and deploy intelligence with speed, accuracy and without the need for specialized skills would be remarkable. I believe that InforSense is that remarkable tool.” -- David Norris, Senior Analyst, Bloor Research Making Workflow Work

Proprietary and Confidential Thank You !