Presentation on theme: "DAS/2: Next Generation Distributed Annotation System Gregg Helt 1, Steve Chervitz 1, Tony Cox 2, Andrew Dalke 3, Allen Day 4, Ed Erwin 1, Ed Griffiths."— Presentation transcript:
DAS/2: Next Generation Distributed Annotation System Gregg Helt 1, Steve Chervitz 1, Tony Cox 2, Andrew Dalke 3, Allen Day 4, Ed Erwin 1, Ed Griffiths 2, and Lincoln Stein 4 (1) Affymetrix, Inc. (2) Sanger Institute (3) Dalke Scientific; (4) Cold Spring Harbor Laboratory (5) University of Alabama
Distributed Annotation System (DAS) Overview A specification designed for sharing genome annotations Defines client requests and server responses Simplified Web Services approach: HTTP GET, URLs, XML Intended to be simple to implement No central annotation authority Intended to support client-side integration of annotations from different servers First draft specification Spring 2000 Last major change to DAS1 was Spring 2002 Grant from NIH awarded June 2004 for development of next-generation DAS/2
DAS: Multiple Servers, Multiple Clients Reference Server AC003027 AC005122 M10154 Annotation Server AC003027 M10154 WI1029AFM820AFM1126WI443 AC005122 Annotation Server
Widespread Adoption of DAS/1 Server Implementations – Dazzle, ProServer, LDAS Server sites – Ensembl, UCSC, TIGR, KEGG, WormBase, Affymetrix, etc. Clients – GBrowse, Ensembl, Dasty, IGB, Libraries: – BioPerl, BioJava, JDAS DAS Extensions – GeneDAS (non-positional annotations) – DAS web services registry – SPICE (protein structures) – DALEC (asynchronous analysis)
Distributed GBrowse My GBrowse GBrowse 1 MODs GBrowse 2 DAS EnsemblUCSC
DAS Limitations No ontology (controlled vocabulary) of feature types. – Is a “gene” from DAS server 1 the same as a “gene” from DAS server 2? Not particularly extensible. Ambiguous semantics for retrieving features that overlap a range on the genome.
Development of DAS/2 Specification Enhancements have largely been motivated by initial discussions on the DAS mailing list. – Series of RFCs collected – Though informal, still a long process! Most recent DAS/2 draft specification is available at http://biodas.org/documents/das2/das2_protocol.ht ml (tied to CVS repository), so anyone can review and comment http://biodas.org/documents/das2/das2_protocol.ht ml Feedback from the DAS developer and user communities will continue to guide future iterations of the DAS/2 specification
Preserving DAS1 Strengths in DAS/2 Specification is independent of implementation – Many server implementations – Many client implementations Simple, simple, simple – HTTP for transport – URLs for queries – XML for responses – REST-like style Ontologies are integral Focus on location-based annotations of biological sequences
Basic DAS/2 Queries Sources query: what genomes and versions of those genomes are available? – http://server/das/genome Regions query: what annotated sequences are available for a given version of a genome? – http://server/das/genome/[genome]/[version]/region Types query: what annotation types are availabe for a given genome version? – http://server/das/genome/[genome]/[version]/type Range query: return all annotations of a given type that overlap a genomic region – http://server/das/genome/[genome]/[version]/feature? overlaps=[seq/min:max];type=[type]
DAS/2 Enhancements: Ontologies All features are required to be described by an ontology – What is the feature? Gene, mRNA, transposable_element… – What are attributes of the feature? Polycistronic_mRNA, programmed_frameshift… Sequence ontology (SO) is the default (song.sourceforge.net) – Can be changed & extended – ~500 terms in all – Standard OBO format Feature hierarchy allows features to be contained within others: e.g. gene->mRNA->CDS
DAS/2 Enhancements: Performance One of the biggest complaints about DAS1 – Very verbose annotation XML DAS/2 Solution #1: Refactoring annotation XML – Much smaller minimum footprint DAS/2 Solution #2: Alternative return formats – All servers can return defined das2xml annotation format – Servers can also specify additional return formats per annotation type – Clients can choose from alternative formats if they desire – Not restricted to XML, or even text – Examples: GFF3, BED, PSL, GAME – Extreme performance improvements possible
DAS/2 Enhancements: Resolving Ambiguities Example: Ambiguous Range Queries query range = x:y xy Server 1 Response: Server 2 Response: Overlap or containment? Parent based or separate? Server 3 Response: Server 4 Response:
DAS/2 Solution #1 – remove spec ambiguity Specify that if parent meets region filter, also return all children Specify whether overlap, containment, etc. Add different region filters for different possibilities – Overlaps – Contains – Within – Identical Allow boolean combinations of these and other filters in the query URL
DAS/2 filter spec allows client query optimization xy QueryL QueryC QueryR LR Keep track of overlap bounds of all previous queries Instead of filter = “overlaps:S/x:y”, use filter = “overlaps:S/x:y; within:S/L:R” If annotation A not contained within L:R, then either: i) bounds crosses L, in which case must overlap QueryL ii) bounds crosses R, in which case must overlap QueryR iii) both Therefore if client has used this approach for all previous queries (and restricts other filtering to single “type” filter), then for QueryC no annotations will be returned that were already returned in a previous query
Solution #2: DAS/2 Validation Suite Verify whether a DAS/2 server is compliant with the specification. – Critical for improving interoperability between clients and servers developed by different groups. Standalone tool and web application, written in Python – Enter a URL for a DAS/2 server – Get an HTML report about DAS/2 compliance Reference dataset – Sequences and annotations that can be loaded into a DAS/2 server for additional validation of server implementation/configuration Source code available at: http://sourceforge.net/projects/dasypus/ http://sourceforge.net/projects/dasypus/
More DAS/2 Spec Enhancements “Writeback” spec to allow DAS/2 clients to create and edit annotations on DAS/2 servers – Still undergoing development IDs are URIs – Could be LSIDs or URLs – Allows for integration with many other web technologies – xml:base Feature hierarchies And more…
DAS/2 Reference Server Implemented as an Apache/mod_perl 2.0 content handler – Annotations are converted to Bioperl objects and subsequently text-transformed using Template Toolkit. Datasources are accessible using an adaptor pattern – Current adapter is for CHADO (GMOD schema) – Soon any datasource accessible to the Generic Genome Browser (Gbrowse) will be be accessible from the DAS/2 server. Flatfile formats: GenBank, GFF Databases: Ensembl, GMOD/Chado, Bio::DB::GFF DAS1 web service Source code released under Artistic License – Available via anonymous CVS as part of GMOD – See http://www.gmod.org for access details.http://www.gmod.org
DAS/2 Reference Client Implemented in Java in the Integrated Genome Browser – IGB (“ig-bee”) - A visualization app developed at Affymetrix – Supports data loading via a variety of formats and mechanisms – Full implementation of DAS/2 read client, partial implementation of DAS/2 writeback. Handles large amounts of genome-scale data – Loads hundreds of thousands of sequence annotations at once – Loads dense quantitative graphs with millions of data points – Maintains real-time responsiveness to user interactions – Includes features to support exploratory data analysis – Plugin architecture for customized extensions Source code released under Common Public License – http://genoviz.sourceforge.net http://genoviz.sourceforge.net
Upcoming DAS/2 Developments Writeback protocol – Ready for implementation Registry and discovery protocol – Various alternatives have been discussed – A “playpen server” available at EBI
DAS/2 & caBIG Project 1: Add DAS/2 support to caCORE – Will enable caCORE to read genome annotations from DAS/2 servers and re-export as caCORE objects. – Uses a flexible plug-in architecture that will be generally useful. Project 2: Export HapMap database as DAS/2 – Will make HapMap human variation data available to caBIG grid via caCORE. Project 3: Export Vertebrate Promoter Database as DAS/2 – Will make curated information on vertebrate transcription factors and their binding sites available to caB IG grid via caCORE.
Acknowledgements DAS & DAS2 mailing list participants! Lincoln Stein (CSHL) Ed Erwin, Steve Chervitz, Eric Blossom, Hari Tammara (Affymetrix) Tony Cox, Ed Griffiths (Sanger Institute) Allen Day, Brian O’Connor (UCLA) Andrew Dalke (Dalke Consulting) Suzanna Lewis (LBL) Ann Loraine (U. of Alabama)
Your consent to our cookies if you continue to use this website.