1. 1 The myGrid Project Professor Chris Greenhalgh University of Nottingham

2. 2 Open Source Upper Middleware for Bioinformatics (Web) Service-based architecture Targeted at Tool Developers, Bioinformaticians and Service Providers Newcastle Nottingham Manchester Southampton Hinxton Sheffield

3. 3 Philosophy Openness –open source –open world of services –open to wider eScience context –open to user feedback –open to third party metadata Collection of components for assembly –Pick and mix

4. 4 Data-intensive bioinformatics ID MURA_BACSU STANDARD; PRT; 429 AA. DE PROBABLE UDP-N-ACETYLGLUCOSAMINE 1-CARBOXYVINYLTRANSFERASE DE (EC 2.5.1.7) (ENOYLPYRUVATE TRANSFERASE) (UDP-N-ACETYLGLUCOSAMINE DE ENOLPYRUVYL TRANSFERASE) (EPT). GN MURA OR MURZ. OS BACILLUS SUBTILIS. OC BACTERIA; FIRMICUTES; BACILLUS/CLOSTRIDIUM GROUP; BACILLACEAE; OC BACILLUS. KW PEPTIDOGLYCAN SYNTHESIS; CELL WALL; TRANSFERASE. FT ACT_SITE 116 116 BINDS PEP (BY SIMILARITY). FT CONFLICT 374 374 S -> A (IN REF. 3). SQ SEQUENCE 429 AA; 46016 MW; 02018C5C CRC32; MEKLNIAGGD SLNGTVHISG AKNSAVALIP ATILANSEVT IEGLPEISDI ETLRDLLKEI GGNVHFENGE MVVDPTSMIS MPLPNGKVKK LRASYYLMGA MLGRFKQAVI GLPGGCHLGP RPIDQHIKGF EALGAEVTNE QGAIYLRAER LRGARIYLDV VSVGATINIM LAAVLAEGKT IIENAAKEPE IIDVATLLTS MGAKIKGAGT NVIRIDGVKE LHGCKHTIIP DRIEAGTFMI

5. 5 Use Scenarios Grave’s Disease Autoimmune disease of the thyroid Simon Pearce and Claire Jennings, Institute of Human Genetics School of Clinical Medical Sciences, University of Newcastle Discover all you can about a gene Annotation pipelines and Gene expression analysis Services from Japan, Hong Kong, various sites in UK Williams-Beuren Syndrome Microdeletion of 155 Mbases on Chromosome 7 Hannah Tipney, May Tassabehji, Andy Brass, St Mary’s Hospital, Manchester, UK Characterise an unknown gene Annotation pipelines and Gene expression analysis Services from USA, Japan, various sites in UK

6. 6 Williams-Beuren Syndrome Microdeletion ** Chr 7 ~155 Mb ~1.5 Mb 7q11.23 GTF2I RFC2CYLN2 GTF2IRD1 NCF1 WBSCR1/E1f4H LIMK1ELNCLDN4CLDN3STX1A WBSCR18 WBSCR21 TBL2BCL7BBAZ1B FZD9 WBSCR5/LAB WBSCR22 FKBP6POM121 NOLR1 GTF2IRD2 C-cen C-midA-cen B-mid B-cen A-midB-telA-telC-tel WBSCR14 WBS SVAS STAG3 PMS2L Block A FKBP6T POM121 NOLR1 Block C GTF2IP NCF1P GTF2IRD2P Block B Patient deletions CTA-315H11 CTB-51J22 Gap Physical Map

7. 7 Manually filling a genomic gap Numerous web-based services (i.e. BLAST, RepeatMasker) Cutting and pasting Large number of steps Frequently repeated – info now rapidly added to public databases Don’t always get results Time consuming Huge amount of interrelated data is produced – handled in lab book and files saved to local hard drive Mundane Much knowledge remains undocumented.: Bioinformatician does the analysis

8. 8 WBS Workflows: GenBank Accession No GenBank Entry Seqret Nucleotide seq (Fasta) GenScanCoding sequence ORFs prettyseq restrict cpgreport RepeatMasker ncbiBlastWrapper sixpack transeq 6 ORFs Restriction enzyme map CpG Island locations and % Repetative elements Translation/sequence file. Good for records and publications Blastn Vs nr, est databases. Amino Acid translation epestfind pepcoil pepstats pscan Identifies PEST seq Identifies FingerPRINTS MW, length, charge, pI, etc Predicts Coiled-coil regions SignalP TargetP PSORTII InterPro PFAM Prosite Smart Hydrophobic regions Predicts cellular location Identifies functional and structural domains/motifs Pepwindow? Octanol? ncbiBlastWrapper URL inc GB identifier tblastn Vs nr, est, est_mouse, est_human databases. Blastp Vs nr RepeatMasker Query nucleotide sequence ncbiBlastWrapper Sort for appropriate Sequences only Pink: Outputs/inputs of a service Purple: Taylor-made services Green: Emboss soaplab services Yellow: Manchester soaplab services Grey: Unknowns RepeatMasker

9. 9 Workflow approach: in-silico experiments Williams-Beuren Syndrome Manually: takes two days (+) including analysis Now takes 30 mins to produce results and half a day for analysis Manually: Do analysis as perform experiment Workflow: Do analysis at end of experiment Therefore need good result co-ordination for back- tracking

10. 10 (e-)Scientists… …Experiment Can workflow be used as an experimental method? How many times has this experiment been run? …Analyze How do we manage the results to draw conclusions from them? How reliable are these results? …Collaborate Can we share workflows, results, metadata etc? …Publish Can we link to these workflows and results from our papers? …Review Can I find, comprehend and review your work? How was that result derived?

11. 11 Web Service (Grid Service) communication fabric AMBIT Text Extraction Service Provenance Personalisation Event Notification Gateway Service and Workflow Discovery myGrid Information Repository Ontology Mgt Metadata Mgt Work bench TavernaTalisman Native Web Services SoapLab Web Portal Legacy apps Registries Ontologies FreeFluo Workflow Enactment Engine OGSA-DQP Distributed Query Processor Bioinformaticians Tool Providers Service Providers Applications Core services External services my Grid Service Stack Views Legacy apps GowLab

12. 12 Web Service (Grid Service) communication fabric AMBIT Text Extraction Service Provenance Personalisation Event Notification Gateway Service and Workflow Discovery myGrid Information Repository Ontology Mgt Metadata Mgt Work bench TavernaTalisman Native Web Services SoapLab Web Portal Legacy apps Registries Ontologies FreeFluo Workflow Enactment Engine OGSA-DQP Distributed Query Processor Bioinformaticians Tool Providers Service Providers Applications Core services External services my Grid Service Stack Views Legacy apps GowLab

13. 13

14. 14 Control flow, iteration and data flow Data sets and nested flows Configurable failure handling Incorporated Life Science Id resolution Provenance and status reporting Type and data management Plug-ins User notification Data entry wizard Libraries of SHIM services Libraries of workflows FreeFluo Features

15. 15 Domain Services Native WSDL Web services –DDBJ, NCBI BLAST, PathPort, BioMOBY Wrapped legacy services –SoapLab –GowLab Web pages as web services –One button wrapping –Leveraged the EMBOSS Suite –~159 services Lots of them and lots of redundant services The joys of firewalls and licensing EBI Support agreed to support Soaplab services as core business http://industry.ebi.ac.uk/soaplab/ For each application CreateJob Run WaitFor GetResults Destroy

16. 16 Two+ Paths Core functionality Services – Soaplab and Gowlab Workflow enactment engine – Freefluo Workflow workbench – Taverna Data integration – OGSADQP Information model & management Innovative work Service and workflow registration Semantic discovery Provenance management Text mining In between Event notification Gateway

17. 17 Drilling Down: my Grid and Semantics Workflow and service discovery –Prior to and during enactment –Semantic registration Workflow assembly –Semantic service typing of inputs and outputs Provenance of workflows and other entities Experimental metadata glue Use of RDF, RDFS, DAML+OIL/OWL –Instance store, ontology server, reasoner –Materialised vs at point of delivery reasoning. my Grid Information Model

18. 18 Workflow run Workflow design Experiment design Project Person Organisation Process Service Event Data item data derivation e.g. output data derived from input data knowledge statements e.g. similar protein sequence to instanceOf partOf componentProcess e.g. web service invocation of BLAST @ NCBI componentEvent e.g. completion of a web service invocation at 12.04pm runBy e.g. BLAST @ NCBI run for Organisation level provenanceProcess level provenance Data/ knowledge level provenance Provenance (1) User can add templates to each workflow process to determine links between data items.

19. 19 19747251AC005089.3 831 Homo sapiens BAC clone CTA-315H11 from 7, complete sequence 15145617AC073846.6 815 Homo sapiens BAC clone RP11-622P13 from 7, complete sequence 15384807AL365366.20 46.1 Human DNA sequence from clone RP11-553N16 on chromosome 1, complete sequence 7717376AL163282.2 44.1 Homo sapiens chromosome 21 segment HS21C082 16304790AL133523.5 44.1 Human chromosome 14 DNA sequence BAC R-775G15 of library RPCI-11 from chromosome 14 of Homo sapiens (Human), complete sequence 34367431BX648272.1 44.1 Homo sapiens mRNA; cDNA DKFZp686G08119 (from clone DKFZp686G08119) 5629923AC007298.17 44.1 Homo sapiens 12q22 BAC RPCI11-256L6 (Roswell Park Cancer Institute Human BAC Library) complete sequence 34533695AK126986.1 44.1 Homo sapiens cDNA FLJ45040 fis, clone BRAWH3020486 20377057AC069363.10 44.1 Homo sapiens chromosome 17, clone RP11-104J23, complete sequence 4191263AL031674.1 44.1 Human DNA sequence from clone RP4-715N11 on chromosome 20q13.1-13.2 Contains two putative novel genes, ESTs, STSs and GSSs, complete sequence 17977487AC093690.5 44.1 Homo sapiens BAC clone RP11-731I19 from 2, complete sequence 17048246AC012568.7 44.1 Homo sapiens chromosome 15, clone RP11-342M21, complete sequence 14485328AL355339.7 44.1 Human DNA sequence from clone RP11-461K13 on chromosome 10, complete sequence 5757554AC007074.2 44.1 Homo sapiens PAC clone RP3-368G6 from X, complete sequence 4176355AC005509.1 44.1 Homo sapiens chromosome 4 clone B200N5 map 4q25, complete sequence 2829108AF042090.1 44.1 Homo sapiens chromosome 21q22.3 PAC 171F15, complete sequence >gi|19747251|gb|AC005089.3| Homo sapiens BAC clone CTA-315H11 from 7, complete sequence AAGCTTTTCTGGCACTGTTTCCTTCTT CCTGATAACCAGAGAAGGAAAAGATC TCCATTTTACAGATGAG GAAACAGGCTCAGAGAGGTCAAGGCT CTGGCTCAAGGTCACACAGCCTGGGA ACGGCAAAGCTGATATTC AAACCCAAGCATCTTGGCTCCAAAGC CCTGGTTTCTGTTCCCACTACTGTCAG TGACCTTGGCAAGCCCT GTCCTCCTCCGGGCTTCACTCTGCAC ACCTGTAACCTGGGGTTAAATGGGCT CACCTGGACTGTTGAGCG urn:lsid:taverna:datathing:15..BLAST_Report rdf:type urn:lsid:taverna:datathing:13..similar_sequences_to.. nucleotide_sequence rdf:type service invocation..created_by workflow invocation workflow definition experiment definition project person group service description organisation..described_by..run_during..invocation_of..part_of..works_for..part_of..author..run_for AB..masked_sequence_of..filtered_version_of Relationship BLAST report has with other items in the repository Other classes of information related to BLAST report RDF Rules

20. 20 Information Model v2 Resources and Identifiers People, teams and organizations Representing the e-science process Experimental methods for e- science Scientific data and the life-science identifier –Types –Identifier Types –Values and Documents Provenance information Annotation and Argumentation In the middle of deployment Bioinformatics middleware – domain neutral

21. 21 LSIDs LSID provides a uniform naming scheme. LSID Resolver guarantees to resolve to same data object. LSID Authority dishes them out. Also returns metadata of object. Used throughout my Grid as an object naming device. my Grid Repository acts an LSID Authority LSID allows universal access to results for collaboration, as well as for review. RDF+LSID explains the context of results, and provides guidance for further investigations. Pioneered by myGrid I3C / IBM / EBI proposal for a Life Science Identifier http://www.i3c.org/wgr/ta/resources/lsid/docs/

22. 22 Using Haystack

23. 23 In a nutshell Pre-Prototype Prototype 1 Experimental Web-based Requirements gathering Architectural workout All services represented NetBeans workbench API-based integration Info Repository oriented XML-based process provenance Workflow enactment engine Prototype 2 Second generation services Reworked information model Open information management Life Science Identifiers RDF based provenance Taverna workbench Web-based portal Demo at ISMB 2003 Full paper and demo at ISMB 2004 GSK deployment Real biology

24. 24 To Dos Improve results management Deployment of mIR Portal for finding workflows, launching & monitoring workflows, launching taverna, browsing results Deploying publicly accessible semantic registry Reinstate service discovery during enactment Large scale data throughput workflow engine Event notification on services Using provenance graphs for impact analysis Hiding LSIDs Lexicons for concept names Hardening semantic discovery Ambient Text Er..Security Etc… “myGrid in a box”

25. 25 Ongoing/Future Activities my Grid-in-a-box Technical follow-ons –Best practice (6) and OMII (Freefluo,Taverna, Event notification) bids Research follow-ons –Semantic Grids, Data Grids, Workflow, Provenance services –PhD students Science follow-ons –Life Sciences: ISPIDER, e-Fungi –Clinical: PsyGrid, CLEF-II –PhD students Networking –LinK-up with BIRN/SEEK/GEON (SDSC) & SCEC/GriPhyN (ISI,USC)

26. 26 Wrap Up Managed the transition from generic middleware development to practical day to day useful services –Real users (plural) fundamental to that End to end support for an entire scenario –A broad view of the e-Science process Show stoppers for practical adoption are not sexy technical showstoppers –Can I incorporate my favourite service? –Can I manage the results? Tapping into (defacto) standards and communities to leverage others results and tools – LSID, Haystack, Pedro… http://www.mygrid.org.uk

27. 27 Acknowledgements myGrid is an EPSRC funded UK eScience Program Pilot Project Particular thanks to the other members of the Taverna project, http://taverna.sf.nethttp://taverna.sf.net

28. 28 myGrid People Core Matthew Addis, Nedim Alpdemir, Tim Carver, Rich Cawley, Neil Davis, Alvaro Fernandes, Justin Ferris, Robert Gaizaukaus, Kevin Glover, Carole Goble, Chris Greenhalgh, Mark Greenwood, Yikun Guo, Ananth Krishna, Peter Li, Phillip Lord, Darren Marvin, Simon Miles, Luc Moreau, Arijit Mukherjee, Tom Oinn, Juri Papay, Savas Parastatidis, Norman Paton, Terry Payne, Matthew Pockock Milena Radenkovic, Stefan Rennick- Egglestone, Peter Rice, Martin Senger, Nick Sharman, Robert Stevens, Victor Tan, Anil Wipat, Paul Watson and Chris Wroe. Users Simon Pearce and Claire Jennings, Institute of Human Genetics School of Clinical Medical Sciences, University of Newcastle, UK Hannah Tipney, May Tassabehji, Andy Brass, St Mary’s Hospital, Manchester, UK Postgraduates Martin Szomszor, Duncan Hull, Jun Zhao, Pinar Alper, John Dickman, Keith Flanagan, Antoon Goderis, Tracy Craddock, Alastair Hampshire Industrial Dennis Quan, Sean Martin, Michael Niemi, Syd Chapman (IBM) Robin McEntire (GSK) Collaborators Keith Decker