Presentation is loading. Please wait.

Presentation is loading. Please wait.

Reese et al., Tutorial #3, ISMB ‘99 The challenge of annotating a complete eukaryotic genome: A case study in Drosophila melanogaster Martin G. Reese

Similar presentations


Presentation on theme: "Reese et al., Tutorial #3, ISMB ‘99 The challenge of annotating a complete eukaryotic genome: A case study in Drosophila melanogaster Martin G. Reese"— Presentation transcript:

1 Reese et al., Tutorial #3, ISMB ‘99 The challenge of annotating a complete eukaryotic genome: A case study in Drosophila melanogaster Martin G. Reese Nomi L. Harris George Hartzell Suzanna E. Lewis Drosophila Genome Center Department of Molecular and Cell Biology 539 Life Sciences Addition University of California, Berkeley

2 Reese et al., Tutorial #3, ISMB ‘99 Abstract Many of the technical issues involved in sequencing complete genomes are essentially solved. Technologies already exist that provide sufficient solutions for ascertaining sequencing error rates and for assembling sequence data. Currently, however, standards or rules for the annotation process are still an outstanding problem. How shall the genomes be annotated, what shall be annotated, which computational tools are most effective, how reliable are these annotations, how organism-specific do the tools have to be and ultimately how should the computational results be presented to the community? All these questions are unsolved. This tutorial will give an overview and assessment of the current state of annotation based upon experiences gained at the Drosophila melanogaster genome project. In the tutorial we will do three things. First, we will break down the annotation process and discuss the various aspects of the problem. This will serve to clarify the term "annotation", which is often used to collectively describe a process that has a number of discrete steps. Second, with the participation of computational biologists from the community we will compare existing tools for sequence annotation. We will do this by providing a 3 megabase sequence that has already been well-characterized at our center as a testbed for evaluating other feature-finding algorithms. This is similar to what has been done at the CASP (critical assessment of techniques for protein structure prediction) conferences (http://predictioncenter.llnl.gov) for protein structure prediction. Third, we will discuss which annotation problems are essentially solved and which problems remain.

3 Reese et al., Tutorial #3, ISMB ‘99 Tutorial goals Review the algorithms currently used in annotation Assess existing methods under “field” conditions Identify open issues in annotation

4 Reese et al., Tutorial #3, ISMB ‘99 Tutorial organization Definitions Annotation  “Biological” issues  “Engineering” issues  Application of tools within an existing annotation system Break (20 minutes) Review of existing tools Our annotation experiment Conclusions and outstanding issues

5 Reese et al., Tutorial #3, ISMB ‘99 What is a gene? Definition: An inheritable trait associated with a region of DNA that codes for a polypeptide chain or specifies an RNA molecule which in turn have an influence on some characteristic phenotype of the organism.

6 Reese et al., Tutorial #3, ISMB ‘99 What are annotations? Definition: Features on the genome derived through the transformation of raw genomic sequences into information by integrating computational tools, auxiliary biological data, and biological knowledge.

7 Reese et al., Tutorial #3, ISMB ‘99 How does an annotation differ from a gene? Many annotations are the same as ‘genes’  The annotation describes an inheritable trait associated with a region of DNA. But an annotation may not always correspond in this way, e.g. an STS, or sequence overlap  Region of genomic DNA or RNA is not translated or transcribed

8 Reese et al., Tutorial #3, ISMB ‘99 Transcription and translation

9 Reese et al., Tutorial #3, ISMB ‘99 Schematic gene structure

10 Reese et al., Tutorial #3, ISMB ‘99 Sequence feature types Transcribed region  mRNA, tRNA, snoRNA, snRNA, rRNA Structural region  Exon, intron, 5’ UTR, 3’ UTR, ORF, cleavage product  Mutations: insertion, deletion, substitution, inversion, translocation  Functional or signal region  Promoter, enhancer, DNA/RNA binding site, splice site signal, poly- adenylation signal  Protein processing: glycosylation, methylation, phosphorylation site Similarity  Homolog, paralog, genomic overlap (syntenic region) Other feature types  Transposable element, repetitive element  Pseudogene  STS, insertion site

11 Reese et al., Tutorial #3, ISMB ‘99 DNA transcription unit features Promoter elements  Core promoter elements  TATA box  Initiator (Inr)  Downstream promoter element (DPE)  Transcription factor (“TF”) binding sites  CAAT boxes  GC boxes  SP-1 sites  GAGA boxes  Enhancer site(s)

12 Reese et al., Tutorial #3, ISMB ‘99 mRNA features Exon  Initial, internal, terminal  Codon usage, preference  Control elements (e.g. splice enhancers) Intron  5’ splice site (“GT”), branchpoint (lariat), 3’ splice site (“AG”)  Repeat elements Start codon (translation start site)  “Kozak” rule UTR (untranslated regions)  5’ UTR  Translation regulatory elements  RNA binding sites  Initial, internal, terminal  Control elements (e.g. splice enhancers)  3’ UTR  RNA binding sites (cis-acting elements) Stop codon Poly-adenylation signal and site RNA destabilization signal

13 Reese et al., Tutorial #3, ISMB ‘99

14 Definitions for data modeling Feature: An interval or an ordered set of intervals on a sequence that describes some biological attribute and is justified by evidence. Sequence: A linear molecule of DNA, RNA or amino acids. Evidence: A computational or experimental result coming out of an analysis of a sequence Annotation: A set of features

15 Reese et al., Tutorial #3, ISMB ‘99 Annotated genome Annotation Depth of knowledge Breadth of knowledge Detailed analysis (typically biological) of single genes Large-scale analysis (typically computational) of entire genome

16 Reese et al., Tutorial #3, ISMB ‘99 Annotation process overview Genome Sequence Auxiliary Data Computational Tools Understanding of a Genome Annotation Systems Data Methods Database Resources

17 Reese et al., Tutorial #3, ISMB ‘99 Types of sequence data Chromosomal sequence  Euchromatic  Heterochromatic mRNA sequences  Full length cDNA  5’ EST  3’ EST Protein sequences Insertion site flanking sequences

18 Reese et al., Tutorial #3, ISMB ‘99 Auxiliary data Maps  Genetic, physical, radiation hybrid map (RH), deletion, cytogenetic Expression data  Tissue, stage Phenotypes  Lethality, sterility

19 Reese et al., Tutorial #3, ISMB ‘99 Computational annotation tools Gene finding Repeat finding EST/cDNA alignment Homology searching  BLAST, FASTA, HMM-based methods, etc. Protein family searching  PFAM, Prosite, etc.

20 Reese et al., Tutorial #3, ISMB ‘99 Database resources Curated sequence feature data sets  Repeat elements  Transposons  Non-redundant mRNA  STSs and other sequence markers Genome sequence from related species  D. melanogaster vs. D. virilis, D. hydei Genome sequence from more distant species Protein sequences from distant species

21 Reese et al., Tutorial #3, ISMB ‘99 Biological issues in annotation Common  Genes within genes  Alternative splicing  Alternative poly-adenylation sites Rare  Translational frame shifting  mRNA editing  Eukaryotic operons  Alternative initiation

22 Reese et al., Tutorial #3, ISMB ‘99 Engineering issues in annotation What sequence to start with?  Because features are intervals on a sequence, problems can be caused by gaps, frameshifts, and other changes to the sequence. How do you track these changes over time and model features that span gaps? When to annotate?  Feature identification can aid in sequencing. It may be advisable to carry out sequencing and annotation in parallel thus enabling them to complement one another. What analyses need to be run and how?  What dependencies are there between various analysis programs?  What parameters settings to use?

23 Reese et al., Tutorial #3, ISMB ‘99 Engineering issues in annotation What public sequence data sets are needed?  What are the mechanics of obtaining public sequence databases?  Are curated data sets available or do you need to set up a means of maintaining your own (for repeats, insertions, organism of interest) How do you achieve computational throughput?  Workstation farm, or simply a big, powerful box?  Job flow control What do you do with the results?  Homogenize results into single format?  Filter results for significance and redundancy

24 Reese et al., Tutorial #3, ISMB ‘99 Engineering issues in annotation Interpreting the results  Is human curation needed?  How can you achieve consistency between curators?  How do you design the user interface so that it is simple enough to get the task completed speedily but complex enough to deal with biology?  How do you capture curations? How are annotation translations to be described?  EC terminology  ProSite families  Pfam domains  Is function distinguishable from process?

25 Reese et al., Tutorial #3, ISMB ‘99 Engineering issues in annotation How do you manage data?  What is the appropriate database schema design?  How is the database to be kept up to date? Will it be directly from programs running user interfaces and analyses or via a middleware layer?  Is a flat file format needed and what should it be?  What query and retrieval support is needed? How do you distribute data?  For bulk downloads what is the format of the data?  What information is best summarized in tables?  What information requires an integrated graphical view?

26 Reese et al., Tutorial #3, ISMB ‘99 Engineering issues in annotation How do you update the annotations?  How frequently are they re-evaluated?  How can re-evaluation be minimized (only subsets of the databanks, only modified sequences)?  How can differences between old and new computational results be detected?  Changes in computational results may need to trigger changes in curated annotations

27 Reese et al., Tutorial #3, ISMB ‘99 Drosophila melanogaster Drosophila is the most important model organism* Drosophila genome:  4 chromosomes  180 Mb total sequence  140 Mb euchromatic sequence  12-14,000 genes * source: G.M. Rubin

28 Reese et al., Tutorial #3, ISMB ‘99 Drosophila Genome Project Laboratories working on Drosophila sequencing:  BDGP (Berkeley Drosophila Genome Project)  EDGP (European Drosophila Genome Project)  Celera Genomics Inc. “Complete” D. melanogaster sequence will be finished by the end of 1999 Comprehensive database - FlyBase

29 Reese et al., Tutorial #3, ISMB ‘99 Goals of the Drosophila Genome Project Complete genome sequence Structure of all transcripts Expression pattern of all genes Phenotype resulting from mutation of all ORFs And more...

30 Reese et al., Tutorial #3, ISMB ‘99 Sequencing at the BDGP Genomic sequence  P1 and BAC clones  24Mb of completed sequence (as of July 22, 1999)  18Mb unfinished sequence in process Complete tiling path in BACs  1.5x-path draft sequencing ESTs and cDNAs  80,942 ESTs finished (as of March 19, 1999)  Over 800 full-length cDNAs

31 Reese et al., Tutorial #3, ISMB ‘99 The BDGP sequence annotation process

32 Reese et al., Tutorial #3, ISMB ‘99 Unit of sequencing at the BDGP  Completed high-quality clone sequences Reassembling the genomic sequence  Need to place clones in correct genomic positions  Need to integrate genes that span multiple clones  Solved by using genomic overlaps to reconstitute full genomic sequence What sequence to start with?

33 Reese et al., Tutorial #3, ISMB ‘99 Which analyses need to be run? Similarity searches  BLAST (Altschul et al., 1990)  BLASTN (nucleotide databases)  BLASTX (amino acid databases)  TBLASTX (amino acid databases, six-frame translation)  sim4 (Miller et al., 1998)  Sequence alignment program for finding near-perfect matches between nucleotide sequences containing introns Gene predictors  Genefinder (Green, unpublished)  GenScan (Burge and Karlin, 1997)  Genie (Reese et al., 1997) Other analyses  tRNAscanSE (Lowe and Eddy, 1996)

34 Reese et al., Tutorial #3, ISMB ‘99 Which analyses need to be run and how? mRNAs  ORFFinder(Frise, unpublished) Protein translations  HMMPFAM 2.1 (Eddy 1998) against PFAM (v Sonnhammer et al. 1997, Bateman et al. 1999)  Ppsearch (Fuchs 1994) against ProSite (release 15.0) filtered with EMOTIF ( Nevill-Manning et al. 1998)  Psort II (Horton and Nakai 1997)  ClustalW (Higgins et al. 1996)

35 Reese et al., Tutorial #3, ISMB ‘99 What public sequence data sets are needed? Automating updates of public databases:  Genbank, SwissProt, trEMBL, BLOCKS, dbEST, EDGP Curated data sets  D. melanogaster genes (FlyBase)  Transposable elements (EDGP)  Repeat elements (EDGP)  STSs (BDGP)

36 Reese et al., Tutorial #3, ISMB ‘99 Which analyses need to be run and how?

37 Reese et al., Tutorial #3, ISMB ‘99 How do you achieve computational throughput? BDGP computing power  Sun Ultra 450 (3 machines, 4 processors each)  Sun Enterprise (1 machine, 8 processors)  Used these directly, without any system for distributed computing. Job flow control: the Genomic Daemon  Automatic batch analysis of genomic clones  Berkeley Fly Database is used for queuing system and storage of results  Many clones can be analyzed simultaneously  Results are processed and saved in XML format for interactive browsing

38 Reese et al., Tutorial #3, ISMB ‘99 What do you do with the results? Berkeley Output Parser (BOP)  Input to BOP:  Genomic sequence  Results of computational analyses  Filtering preferences  Parses results from BLAST, sim4, GeneFinder, GenScan, and tRNAscan-SE analyses  Filters BLAST and sim4 results  Eliminates redundant or insignificant hits  Merges hits that represent single region of homology  Homogenizes results into single format  Output: sequence and filtered results in XML format

39 Reese et al., Tutorial #3, ISMB ‘99 Is human curation needed? Not for everything  Some features are obvious and can be identified computationally  Known D. melanogaster genes are detected automatically by GeneSkimmer  Repetitive elements But still for many things  Annotating complete gene structure is still hard  We use CloneCurator (BDGP’s Java graphical editor) for curation

40 Reese et al., Tutorial #3, ISMB ‘99 Gene Skimmer Quick way of identifying genes in new sequence before curation Start with XML output from BOP Look for sim4 hits with known Drosophila genes Find gene hits with sequence identity >98%, coverage >30% Verify that hits represent real genes

41 Reese et al., Tutorial #3, ISMB ‘99 Gene Skimmer URL:

42 Reese et al., Tutorial #3, ISMB ‘99 CloneCurator Displays computational results and annotations on a genomic clone Interactive browsing  Zoom/scroll  Change cutoffs for display of results  Analyze GC content, restriction sites, etc. Interactive annotation editing  Expert “endorses” selected results Presents annotations to community via Web site

43 Reese et al., Tutorial #3, ISMB ‘99

44 How do we annotate gene/protein function? Gene Ontology Project  Controlled hierarchical vocabulary for multiple-genome annotations and comparisons  Standardized vocabulary facilitates collaboration  Good data modeling allows better database querying  Ontology browser provides interactive search of hierarchical terms  “GO” project (http://www.ebi.ac.uk/~ashburn/GO)

45 Reese et al., Tutorial #3, ISMB ‘99 Ontology browser

46 Reese et al., Tutorial #3, ISMB ‘99

47 Ontology browser: searching for terms

48 Reese et al., Tutorial #3, ISMB ‘99 How do you distribute the data? Bulk downloads  FASTA at  Curated data sets Tabular data  At  Sequenced genomic clones  Clone contigs sorted by genomic location  Clone contigs sorted by size Ribbon provides integrated graphical view of annotations on physical contigs

49 Reese et al., Tutorial #3, ISMB ‘99 Ribbon Human curator annotates individual clones (~100Kb) Clones are assembled into physical contigs (regions of physical map) Clone annotations are merged and renumbered for display on whole physical contigs Ribbon is our Java display tool for displaying curated annotations on physical contigs Will soon be available on Web

50 Reese et al., Tutorial #3, ISMB ‘99 Ribbon

51 Reese et al., Tutorial #3, ISMB ‘99 How do you manage the data? Using Informix as our database server Updated via Perl dbi.pm module Development underway in  Schema revisions  GAME DTD (Genome Annotation Markup Entities)  Perl module for annotation objects  (Ewan Birney)

52 Reese et al., Tutorial #3, ISMB ‘99 How do you maintain annotations? Open questions  How frequently are annotations re-evaluated?  How can re-evaluation be minimized (only subsets of the databanks, only modified sequences)?  How can differences between old and new computational results be detected?  Changes in computational results may need to trigger changes in curated annotations

53 Reese et al., Tutorial #3, ISMB ‘99 Integrated annotation systems ACeDB Genotator Magpie GAIA TIGR

54 Reese et al., Tutorial #3, ISMB ‘99 Integrated annotation systems: ACeDB Developed for analysis of the C. elegans genome Sophisticated database designed for storing annotations and related information New Java and Web-based versions available Written by Jean Thierry-Mieg and Richard Durbin

55 Reese et al., Tutorial #3, ISMB ‘99 ACeDB

56 Reese et al., Tutorial #3, ISMB ‘99 Genotator Back end automates sequence analysis; browser provides interactive viewing and editing of annotations Nomi Harris (1997), Genome Research 7(7),

57 Reese et al., Tutorial #3, ISMB ‘99 Magpie Expert system based (PROLOG)  Data collection daemon  Data analysis and report daemon “Intelligent” integration of various individual feature prediction systems Allows human interactions Gaasterlund and Sensen (1996), TIG, 12,

58 Reese et al., Tutorial #3, ISMB ‘99 GAIA Web-based system Results displayed as Java applets Bailey, L.C., J. Schug, S. Fischer, M. Gibson, J. Crabtree, D.B. Searls, and G.C. Overton (1998), Genome Research.

59 Reese et al., Tutorial #3, ISMB ‘99 Gene Indices for various organisms Databases for transcribed genes linked into external/internal genomic databases Internal backend analysis software TIGR Human Gene Index

60 Reese et al., Tutorial #3, ISMB ‘99 Computational analysis tools Gene finding Repeat finding EST/cDNA alignment Homology searching  BLAST, FASTA, HMM-based methods, etc. Protein family searching  PFAM, Prosite, etc.

61 Reese et al., Tutorial #3, ISMB ‘99 Gene finding: Prokaryotes vs. Eukaryotes Prokaryotes  Contiguous open reading frames (ORF)  Short intergenic sequences  Good method: detecting large ORFs  Complications:  Partial sequences  Sequencing errors  Start codon prediction  Overlapping genes on both strands

62 Reese et al., Tutorial #3, ISMB ‘99 Gene finding: Prokaryotes vs. Eukaryotes Eukaryotes  Complex gene structures (exon/introns)  D. melanogaster has an average of 4 introns/gene  Very long genes (D. melanogaster X gene 160 kb)  Very long introns  Many introns  “Nested”, overlapping, and alternatively spliced genes  5’ UTRs with non-coding exons  Long 3’ UTRs  Complex transcription machinery  ORF-finding alone is not adequate

63 Reese et al., Tutorial #3, ISMB ‘99 Integrated gene finding Assumptions  Signals and content method sensors alone are not sufficient for predicting gene structure  Gene structure is hierarchical  Each component (exon, intron, splice site, etc.) can be modeled independently The approach  Generate a list of candidates for each component (with scores)  Assemble the components into a “gene model”

64 Reese et al., Tutorial #3, ISMB ‘99 Integrated gene finding: Dynamic programming Determines the best combination of components Two-part problem:  Develop an “optimal” scoring function  Use dynamic programming to find an “optimal” alignment through scoring matrix

65 Reese et al., Tutorial #3, ISMB ‘99 Integrated gene finding: Dynamic programming

66 Reese et al., Tutorial #3, ISMB ‘99 Integrated gene finding: Linear and Quadratic Discriminant Analysis (LDA/QDA) LDA  Deterministic calculation of thresholds  n-class discrimination  Example:  HSPL, Solovyev et al. (1997), ISMB, 5, QDA  Can represent a great improvement over LDA  Example:  MZEF, Michael Zhang (1997), PNAS, 94,

67 Reese et al., Tutorial #3, ISMB ‘99 Integrated gene finding: Feed-forward neural networks Supervised learning Training to discriminate between several feature classes Computing units Gradient descent optimization Multi-layer networks Limitations  Black-box predictions  Local minima Example:  GRAIL, Uberbacher et al. (1991), PNAS, 88,

68 Reese et al., Tutorial #3, ISMB ‘99 Approaches to gene finding: Hidden Markov models Model  A finite model describing a probability distribution over all possible sequences of equal length  “Natural” scoring function  (Conditional) Maximum likelihood “training” Markov  k-order Markov chain: current state dependent on k previous states  The next state in a 1st-order Markov model depends on current state Hidden  Hidden states generate visible symbols Assumptions  Independence of states  No long range correlation Example: HMMgene, A. Krogh (1998), In Guide to Human Genome Computing,

69 Reese et al., Tutorial #3, ISMB ‘99 Approaches to gene finding: Generalized hidden Markov models Each HMM state can be a probabilistic sub-model Complex hierarchical system Requires care in modeling state overlaps Example:  Genie, Kulp et al. (1996), ISMB, 4,  GenScan, Burge and Karlin (1997), JMB, 268(1), 78-94

70 Reese et al., Tutorial #3, ISMB ‘99 Gene finding software Signal recognition  Promoter prediction  Splice site prediction  Start codon prediction  Poly-adenylation site prediction Coding potential Coding exons Gene structure prediction  Spliced alignment  LDA/QDA  Neural networks  HMMs and GHMMs

71 Reese et al., Tutorial #3, ISMB ‘99 Promoter recognition PromoterScan  Identify potential promoter regions  Based on databases of known TF binding sites  TFD (Gosh (1991), TIBS, 16, )  TRANSFAC (Heinemeyer et al. (1999), NAR, 27, )  Prestridge (1995), JMB, 249,  MatInd and MatInspector  Finding consensus matches to known TF binding sites  Based on TRANSFAC  Heinemeyer et al. (1999), NAR, 27,  Quandt et al. (1995), NAR, 23, 

72 Reese et al., Tutorial #3, ISMB ‘99 Promoter recognition (cont.) TSSG/TSSW  LDA based combination of several features (TATA-box, Inr signal, upstream regions)  Solovyev et al. (1997), ISMB, 5,  Transcription Element Search Software  Identify TF binding sites  Based on TRANSFAC 

73 Reese et al., Tutorial #3, ISMB ‘99 Promoter recognition (cont.) CBS Promoter 2.0 Prediction Server  Simulated transcription factors  Principles common to neural networks and genetic algorithms  Knudsen (1999), Bioinformatics 13(5),  CorePromoter  Position dependent 5-tuple  QDA  Michael Zhang (1998), Genome Research, 8, 

74 Reese et al., Tutorial #3, ISMB ‘99 Promoter recognition (cont.) Neural network promoter prediction (NNPP)  Time-delay neural network  Combining TATA box and initiator  Reese (1999), in preparation. 

75 Reese et al., Tutorial #3, ISMB ‘99 Example: NNPP

76 Reese et al., Tutorial #3, ISMB ‘99 Promoter recognition (cont.) Markov chain promoter finder  Competing interpolated Markov chains for promoters, exons, introns  Promoter model consists of five states representing the core promoter parts  Ohler, Reese et al., Bioinformatics 13(5),

77 Reese et al., Tutorial #3, ISMB ‘99 Splice site prediction Nakata, 1985  Nakata (1985), NAR, 13(14), BCM GeneFinder  HSPL - Prediction of splice sites in human DNA sequences  Triplet frequencies in various functional parts of splice site regions  Combined with codon statistics  Solovyev et al. (1994), NAR, 22(24), 

78 Reese et al., Tutorial #3, ISMB ‘99 Splice site prediction (cont.) Neural Network splice site predictor (NNSPLICE)  Multi-layered feed-forward neural network  Modeled after Brunak et al. (1991), JMB, 220,  Reese et al. (1997), JCB, 4(3),  NetGene2  Combination of neural networks and rule-based system  Splice site signal neural network combined with coding potential  Hebsgaard et al. (1996), NAR, 24(17),  Brunak et al. (1991), JMB, 220, 

79 Reese et al., Tutorial #3, ISMB ‘99 Splice site prediction (cont.) SplicePredictor  Logitlinear models for splice site regions  Degree of matching to the splice site consensus  Local compositional contrast  Brendel and Kleffe (1998), NAR, 26(20), 

80 Reese et al., Tutorial #3, ISMB ‘99 Start codon prediction NetStart  Trained on cDNA-like sequences  Neural network based  Local start codon information  Global sequence information  Pedersen and Nielsen (1997), ISMB, 5, 

81 Reese et al., Tutorial #3, ISMB ‘99 Poly-adenylation signal prediction BCM GeneFinder  POLYAH - Recognition of 3'-end cleavage and poly- adenylation region  Triplet frequencies in various functional parts in poly- adenylation regions  LDA  Solovyev et al. (1994), NAR, 22(24), 

82 Reese et al., Tutorial #3, ISMB ‘99 Prediction of coding potential Periodicity detection  Coding sequences have an inherent periodicity of three  Especially good on long coding sequences  Auto-correlation  Seeking the strongest response when shifted sequence is compared with original  Michel (1986), J. Theor. Biol. 120,  Fourier transformation: Spectral analysis  Detection of peak at position corresponding to 1/3 of the frequency  Silverman and Linsker (1986), J. Theor. Biol. 118,

83 Reese et al., Tutorial #3, ISMB ‘99 Prediction of coding potential (cont.) Trifonov (1980;1987)  G-notG-U periodicity  JMB, 194, Fickett (1982)  Position asymmetry in the three codon positions  NAR 10(17), Staden (1984)  Codon usage in tables  NAR 12,

84 Reese et al., Tutorial #3, ISMB ‘99 Prediction of coding potential (cont.) Claverie and Bougueleret (1987)  Hexamer frequency differentials  NAR 14, Fichant and Gautier (1987)  Codon usage homogeneity  CABIOS, 3(4), GRAIL I (1991)  Neural network using a shifting fixed size window  7 sensors as input, 2 hidden layers and 1 unit as output  Uberbacher et al. (1991), PNAS, 88(24),

85 Reese et al., Tutorial #3, ISMB ‘99 Prediction of coding potential (cont.) GeneMark (1986)  Inhomogeneous Markov chain models  Easy trainable (closed solution for Maximum Likelihood)  Used extensively in prokaryotic genomes  Borodovsky et al. (1993), Computers & Chemistry, 17, Glimmer (1998)  Interpolated Markov chains from first to eighth order  Salzberg et al. (1998), NAR, 26(2), 

86 Reese et al., Tutorial #3, ISMB ‘99 Prediction of coding potential (cont.) Review by Fickett (1992)  “Assessment of protein coding measures”, NAR, 20,

87 Reese et al., Tutorial #3, ISMB ‘99 Prediction of coding exons SorFind  Detection of “spliceable” ORFs  Hutchinson, NAR, 20(13), BCM GeneFinder  FEXD, FEXN, FEXA, FEXY, FEXH, HEXON  LDA  Solovyev et al. (1994), NAR, 22(24),  GRAIL II  Exon candidates, heuristic integration, learning with neural network  Uberbacher et al., Genet. Eng., 16, 

88 Reese et al., Tutorial #3, ISMB ‘99 “Integrated” gene models: LDA/QDA FGene  LDA based  Dynamic programming for the integration of LDA output  Solovyev et al. (1995), ISMB, 3, 

89 Reese et al., Tutorial #3, ISMB ‘99 “Integrated” gene models: NN GeneParser  “Gene-parsing” approach  Potential alternative splicing recognized  Neural network and dynamic programming  Snyder and Stormo (1995), JMB, 248, 1-18.

90 Reese et al., Tutorial #3, ISMB ‘99 “Integrated” gene models: Artificial intelligence approaches GeneID  Rule-based system  Homology integration  Guigó et al. (1992), JMB, 226,  GeneID using DP  DP to combine a set of potential exons  Guigó et al. (1998), JCB, 5,

91 Reese et al., Tutorial #3, ISMB ‘99 “Integrated” gene models: Artificial intelligence approaches GenLang  Syntactic pattern recognition system  Formal grammar  Tools from computational linguistics  Dong and Searls (1994), Genomics, 23, 

92 Reese et al., Tutorial #3, ISMB ‘99 HMMGene  Several genes per sequence possible  User constraints possible  Krogh (1997), ISMB, 5,  GeneMark.hmm  Based on GeneMark program for bacterial sequences  Can predict frame shifts  Trained for various organisms  Lukashin and Borodovsky (1998), NAR, 26,  “Integrated” gene models: HMMs

93 Reese et al., Tutorial #3, ISMB ‘99 “Integrated” gene models: GHMMs Genie  Generalized hidden Markov model with length distribution  Integration of multiple content and signal sensors  Content: codon statistics, repeats, intron, intergenic, database homology hits  Signal: promoter, start codon, splice sites, stop codon  Dynamic programming to find optimal parse  Several genes per sequence possible  Kulp et al. (1996), ISMB, 4,  Reese et al. (1997), JCB, 4(3), 

94 Reese et al., Tutorial #3, ISMB ‘99 Example: Genie

95 Reese et al., Tutorial #3, ISMB ‘99 “Integrated” gene models: GHMMs GenScan  Multiple content and signal models  Semi-hidden Markov model sensors with length distribution  Takes GC content into account (separate models)  Several genes per sequence possible  Burge and Karlin (1997), JMB, 268(1), 

96 Reese et al., Tutorial #3, ISMB ‘99 EST/cDNA alignment for gene finding: Spliced alignments PROCRUSTES  Spliced alignment algorithm  Dynamic programming to combine a set of potential exons  Frame conservation  Homologous sequence needed  Gelfand et al. (1996), PNAS, 93, 

97 Reese et al., Tutorial #3, ISMB ‘99 EST/cDNA alignment Sim4  Aligns cDNA to genomic sequence  Uses local similarity  Florea et al. (1998), Genome Research, 8, GeneWise  Dynamic programming  Partial genes allowed  Based on Pfam and statistical splice site models  Birney (1999), unpublished 

98 Reese et al., Tutorial #3, ISMB ‘99 EST/cDNA alignment (cont.) ACEMBLY  Aligns ESTs to genomic sequence  Identifies alternative splicing  Integrated in ACeDB  Jean Thierry-Mieg (unpublished)

99 Reese et al., Tutorial #3, ISMB ‘99 Repeat finders Censor  Uses database of repeat sequences  Jurka et al. (1996), Comp. and Chem., 20(1), BLAST  Integrated masking operations  XBLAST procedure  Claverie (1994), In Automated DNA Sequencing and Analysis Techniques, M. D. Adams, C. Fields and J. C. Venter, eds.,  http//:www.ncbi.nlm.nih.gov/BLAST

100 Reese et al., Tutorial #3, ISMB ‘99 Repeat finders (cont.) RepeatMasker  Detection of interspersed repeats  Smit and Green, unpublished results 

101 Reese et al., Tutorial #3, ISMB ‘99 Homology searching BLAST suite  BLASTN, BLASTX, TBLASTX, PSI-BLAST  Altschul et al. (1990), JMB, 215,  FASTA suite  FASTA, TFASTA  Pearson and Lipman (1988), PNAS, 85, HMM-based searching  SAM (UCSC group)   HMMER, Sean Eddy 

102 Reese et al., Tutorial #3, ISMB ‘99 Gene family searching BLOCKS  PROSITE  PFAM  SCOP 

103 Reese et al., Tutorial #3, ISMB ‘99 The genome annotation experiment (GASP1) Genome Annotation Assessment Project (GASP1) Annotation of 2.9 Mb of Drosophila melanogaster genomic DNA Open to everybody, announced on several mailing lists Participants can use any analysis methods they like (gene finding programs, homology searches, by-eye assessment, combination methods, etc.) and should disclose their methods. “CASP” like 12 participating groups

104 Reese et al., Tutorial #3, ISMB ‘99 URL:

105 Reese et al., Tutorial #3, ISMB ‘99 Goals of the experiment Compare and contrast various genome annotation methods Objective assessment of the state of the art in gene finding and functional site prediction Identify outstanding problems in computational methods for the annotation process

106 Reese et al., Tutorial #3, ISMB ‘99 Adh contig 2.9 Mb contiguous Drosophila sequence from the Adh region, one of the best studied genomic regions  From chromosome 2L (34D-36A)  Ashburner et al., (to appear in Genetics)  222 gene annotations (as of July 22, 1999)  375,585 bases are coding (12.95%) We chose the Adh region because it was thought to be typical. A representative test bed to evaluate annotation techniques.

107 Reese et al., Tutorial #3, ISMB ‘99 Adh paper (to appear in Genetics) URL:

108 Reese et al., Tutorial #3, ISMB ‘99 Raw sequence: Adh.fa GAATTCCCGGTTCAATCTCGTAGAACTTGCCCTTGGTGGACAGTGGGACGTACAACACCTGCCGGTTTTCATTAAGCAGCTGGGCA TACTTCTTTTCCTTCTCCCTTCCCATGTACCCACTGCCATGGGACCTGGTCGCATTGCCGTTGCCATGTTGCGACATATTGACCTG ATCCTGTTTGCCATCCTCGAAGACGGCCAACAGACGGAATACCTGCCCGCCCCTTGCCGTCGTTTTCACGTACTGTGGTCGTCCCT TGTTTATGGGCAGGCATCCCTCGTGCGTTGGACTGCTCGTACTGTTGGGCGAGGATTCCGTAAACGCCGGCATGTTGTCCACTGAG ACAAACTTGTAAACCCGTTCCCGAACCAGCTGTATCAGAGATCCGTATTGTGTGGCCGTGGGGAGACCCTTCTCGCTTAGCATCGA AAAGTAACCTGCGGGAATTCCACGGAAATGTCAGGAGATAGGAGAAGAAAACAGAACAACAGCAAATACTGAGCCCAAATGAGCGA TAGATAGATAGATCGTGCGGCGATCTCGTACTGGTAACTGGTAATTTGATCGATTCAAACGATTCTGGGTCTCCCCGGTTTTCTGG TTCTGGCTTACGATCGGGTTTTGGGCTTTGGTTGTGGCCTCCAGTTCTCTGGCTCGTTGCCTGTGCCAATTCAAGTGCGCATCCGG CCGTGTGTGTGGGCGCAATTATGTTTATTTACTGGTAACTGGTAATTTGATCGATTCAAACGATTCTGGGTCTCCCCGGTTTTCTG TCCCGGTTCAATCTCGTAGAACTTGCCCTTGGTGGACAGTGGGACGTACAACACCTGCCGGTTTTCATTAAGCAGCTGGGCATACT TCTTTTCCTTCTCCCTTCCCATGTACCCACTGCCATGGGACCTGGTCGCATTGCCGTTGCCATGTTGCGACATATTGACCTGATCC TGTTTGCCATCCTCGAAGACGGCCAACAGACGGAATACCTGCCCGCCCCTTGCCGTCGTTTTCACGTACTGTGGTCGTCCCTTGTT AAAGTAACCTGCGGGAATTCCACGGAAATGTCAGGAGATAGGAGAAGAAAACAGAACAACAGCAAATACTGAGCCCAAATGAGCGA TAGATAGATAGATCGTGCGGCGATCTCGTACTGGTAACTGGTAATTTGATCGATTCAAACGATTCTGGGTCTCCCCGGTTTTCTGG TTCTGGCTTACGATCGGGTTTTGGGCTTTGGTTGTGGCCTCCAGTTCTCTGGCTCGTTGCCTGTGCCAATTCAAGTGCGCATCCGG CCGTGTGTGTGGGCGCAATTATGTTTATTTACTGGTAACTGGTAATTTGATCGATTCAAACGATTCTGGGTCTCCCCGGTTTTCTG TCCCGGTTCAATCTCGTAGAACTTGCCCTTGGTGGACAGTGGGACGTACAACACCTGCCGGTTTTCATTAAGCAGCTGGGCATACT TCTTTTCCTTCTCCCTTCCCATGTACCCACTGCCATGGGACCTGGTCGCATTGCCGTTGCCATGTTGCGACATATTGACCTGATCC TGTTTGCCATCCTCGAAGACGGCCAACAGACGGAATACCTGCCCGCCCCTTGCCGTCGTTTTCACGTACTGTGGTCGTCCCTTGTT TATGGGCAGGCATCCCTCGTGCGTTGGACTGCTCGTACTGTTGGGCGAGGATTCCGTAAACGCCGGCATGTTGTCCACTGAGACAA ACTTGTAAACCCGTTCCCGAACCAGCTGTATCAGAGATCCGTATTGTGTGGCCGTGGGGAGACCCTTCTCGCTTAGCATCGAAAAG CTTACGATCGGGTTTTGGGCTTTGGTTGTGGCCTCCAGTTCTCTGGCTCGTTGCCTGTGCCAATTCAAGTGCGCATCCGGCCGTGT GTGTGGGCGCAATTATGTTTATTTACTGGTAACTGGTAATTTGATCGATTCAAACGATTCTGGGTCTCCCCGGTTTTCTGTCCCGG TTCAATCTCGTAGAACTTGCCCTTGGTGGACAGTGGGACGTACAACACCTGCCGGTTTTCATTAAGCAGCTGGGCATACTTCTTTT CCTTCTCCCTTCCCATGTACCCACTGCCATGGGACCTGGTCGCATTGCCGTTGCCATGTTGCGACATATTGACCTGATCCTGTTTG ACTGGTAACTGGTAATTTGATCGATTCAAACGATTCTGGGTCTCCCCGGTTTTCTGTCCCGGTTCAATCTCGTAGAACTTGCCCTT GGTGGACAGTGGGACGTACAACACCTGCCGGTTTTCATTAAGCAGCTGGGCATACTTCTTTTCCTTCTCCCTTCCCATGTACCCAC TGCCATGGGACCTGGTCGCATTGCCGTTGCCATGTTGCGACATATTGACCTGATCCTGTTTGCCATCCTCGAAGACGGCCAACAGA CGGAATACCTGCCCGCCCCTTGCCGTCGTTTTCACGTACTGTGGTCGTCCCTTGTTTATGGGCAGGCATCCCTCGTGCGTTGGACT GCTCGTACTGTTGGGCGAGGATTCCGTAAACGCCGGCATGTTGTCCACTGAGACAAACTTGTAAACCCGTTCCCGAACCAGCTGTA TCAGAGATCCGTATTGTGTGGCCGTGGGGAGACCCTTCTCGCTTAGCATCGAAAAGTAACCTGCGGGAATTCCACGGAAATGTCAG GAGATAGGAGAAGAAAACAGAACAACAGCAAATACTGTGCGGCGATCTCGTACTGGACGGAAATGTCAGGAGATAGGAGAAGAAAA

109 Reese et al., Tutorial #3, ISMB ‘99 Drosophila data sets provided to participants Curated Drosophila nuclear DNA "coding sequences" (CDS) Curated non-redundant Drosophila genomic DNA data (275 “multi”- and 144 “single”-exon sequence entries from Genbank) Drosophila 5' and 3' splice sites Drosophila start codon sites Drosophila promoter sequences Drosophila repeat sequences Drosophila transposon sequences Drosophila cDNA sequences Drosophila EST sequences URL:

110 Reese et al., Tutorial #3, ISMB ‘99 Timetable May 13, June 30, 1999  Distribution of the sample sequence and associated data to the predictors. Collection of predictions. June 30, July 31, 1999  Evaluation of the predictions by the Drosophila Genome Center. August 4, 1999  External expert assessment of the prediction results (HUGO meeting, EMBL) August 6, 1999  Tutorial #3 at the ISMB ‘99 conference in Heidelberg, Germany

111 Reese et al., Tutorial #3, ISMB ‘99 Resources for assessing predictions 80 cDNA sequences NOT in Genbank before experiment deadline  Sequenced from 5 different cDNA libraries  3 paralogs to other genes in the genome  19 cDNAs with cloning artifacts  2 apparently representing unspliced RNA  Multiple inserts (2 cDNAs cloned in the same vector)  58 “usable” cDNAs 33 cDNA sequences in Genbank during experiment Annotations from Adh paper

112 Reese et al., Tutorial #3, ISMB ‘99 Curated data sets for assessing predictions Standard 1 (Adh.std1.gff) “conservative gene set”  43 gene structures (7 single- and 36 multi- coding exon genes)  Criteria for inclusion:  >=95% (most >=99%) of the cDNA aligned to genomic DNA (using sim4)  “GT”/”AG” splice site consensus sequences  Splice site score from neural net 5’ splice sites: >=0.35 threshold ( 98% True Positive score) 3’ splice sites: >=0.25 threshold ( 92% True Positive score)  Start codon and stop codon annotations from Standard 3 (derived from Adh paper)  These 43 genes represent “typical” genes

113 Reese et al., Tutorial #3, ISMB ‘99 Curated data sets for assessing predictions Standard 2 (Adh.std2.gff)  Superset of Standard 1  15 additional gene structures  Same alignment criteria as Standard 1 but no splice site consensus requirement  Not used in the experiment

114 Reese et al., Tutorial #3, ISMB ‘99 Curated data sets for assessment Standard 3 (Adh.std3.gff) “more complete gene set”  222 gene structures (39 single- and 183 multi- coding exon genes)  Criteria:  Annotated as described in Ashburner et al.  cDNA to genomic alignment using sim4  Start codons predicted by ORFFinder (Frise et al., unpublished)  ~182 genes have similarity to a homologous protein sequence in another organism or have a Drosophila EST hit Edge verification by partial EST/cDNA alignments BLASTX, TBLASTX homology results PFAM alignments Gene structure verification using GenScan (human)  14 genes had EST/homology hits but no gene finding predictions  ~40 genes only have “strong” GenScan predictions

115 Reese et al., Tutorial #3, ISMB ‘99 Submission format GFF (Durbin and Haussler, 1998, unpublished) 

116 Reese et al., Tutorial #3, ISMB ‘99 # organism: Drosophila melanogaster # std1 Adh std1 TFBS Adh std1 TATA_signal transcript "1" Adh std1 TSS transcript "1" Adh std1 prim_transcript transcript "1" Adh std1 exon transcript "1" Adh std1 start_codon transcript "1" Adh std1 CDS transcript "1" Adh std1 splice transcript "1" Adh std1 splice transcript "1" Adh std1 exon transcript "1" Adh std1 CDS transcript "1" Adh std1 splice transcript "1" Adh std1 splice transcript "1" Adh std1 CDS transcript "1" Adh std1 exon transcript "1" Adh std1 stop_codon transcript "1" Adh std1 polyA_signal transcript "1" Adh std1 polyA_site transcript "1" Adh std1 prim_transcript transcript "2" Adh std1 exon transcript "2" Adh std1 polyA_site transcript "2" Adh std1 polyA_signal transcript "2" Adh std1 stop_codon transcript "2" Adh std1 CDS transcript "2" Adh std1 start_codon transcript "2" Adh std1 TSS transcript "2" Adh std1 TATA_signal transcript "2" Adh std1 TFBS Adh std1 TFBS Sample submission Gene 1 Gene 2

117 Reese et al., Tutorial #3, ISMB ‘99 Submissions MAGPIE Team  Credit  Terry Gaasterland, Alexander Sczyrba, Elizabeth Thomas, Gulriz Kurban, Paul Gordon, Christoph Sensen  Laboratory for Computational Genomics, Rockefeller and Institute for Marine Biosciences, Canada  Method  Automatic genome analysis system integrating Drosophila Genscan predictions, confirming exons boundaries using database searches, repeat finding (Calypso, REPupter) and gene function annotations.

118 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.)  References  “Multigenome MAGPIE” poster at ISMB ‘99.  Gaasterland and Ragan (1998), J. of Microbial and Comparative Genomics, 3,  Gaasterland and Sensen (1996), Biochimie 78,  REPupter: Kurtz and Schleiermacher (1999), Bioinformatics 15(5),

119 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.) Computational Genomics Group, The Sanger Centre  Credit  Victor Solovyev, Asaf Salamov  Method  Discriminant analysis based gene prediction programs FGenes (trained for Human) and FGenesH (trained for Drosophila); Combining the output of Fgenes, FGenesH and BLAST using FGenesH+. 3 different “threshold” annotations are submitted.  The programming running time is linear with the sequence length.  Automatic, plus additional user interactive screening.  Non-redundant NCBI database used for BLAST.  URL/References 

120 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.) Genome Annotation Group, The Sanger Centre  Credit  Ewan Birney  Method  Protein family based gene identification using Wise2 (previously Genewise) and PFAM.  URL 

121 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.) Pattern Recognition, The University of Erlangen  Credit  Uwe Ohler, Georg Stemmer, Stefan Harbeck, Heinrich Niemann  Method  Promoter recognition based on interpolated Markov chains; “Genscan” like promoter model (MCPromoter); maximal mutual information based estimation of interpolated Markov chains.  Automatic.  Promoter training data set from

122 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.)  References  Ohler, Harbeck, Niemann, Noeth and Reese (1999), Bioinformatics 15(5),  Ohler, Harbeck and Niemann (1999), Proc. EUROSPEECH, to appear.  URL 

123 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.) Computational Biosciences, Oakridge National Laboratory  Credit  Richard J. Mural, Douglas Hyatt, Frank Larimer, Manesh Shah, Morey Parang  Method  Integrated neural network based system including gene assembly using EST and homology information (GRAILexp).  URL: 

124 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.) Center for Biological Sequence Analysis, Technical University of Denmark  Credit  Anders Krogh  Method  Modular HMM incorporating database hits (proteins and ESTs/cDNAS) and other “external information” probabilistically (HMMGene); the HMM has modules for coding regions, splice sites, translation start/stop, etc..  It will be a fully automated system.  Trained on Drosophila data  and Victor Solovyev (personal communication)

125 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.)  References  Krogh (1998), In S.L. Salzberg et al., eds., Computational Methods in Molecular Biology, 45-63, Elsevier.  Krogh (1997), Gaasterland et al., eds., Proc. ISMB 97,   URL   Not yet for Drosophila.

126 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.) BLOCKS group, Fred Hutchinson Cancer Research Center in Seattle, Washington  Credit  Jorja Henikoff, Steve Henikoff  Method  DNA translation in 6 frames and search against BLOCKS+ and against BLOCKS extracted from Smart3.0 (http://coot-embl- heidelberg.de/SMART/) using BLIMPS; automatic post-processing to join multiple predictions from the same block.  Automatic with some user interactive screening of results.

127 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.)  References  Henikoff, Henikoff and Pietrokovski (1999), Nucl. Acids Res., 27,  Henikoff and Henikoff (1994), Proc. 27th Ann. Hawaii Intl. Conf. On System Sciences,  Henikoff and Henikoff (1994), Genomics, 19,  URL  

128 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.) Genome Informatics Team, IMIM, Barcelona, Spain  Credit  Roderic Guigó, Josep F. Abril, Enrique Blanco, Moises Burset, Genis Parra  Method  Dynamic programming based system to combine potential exon candidates modeled as a fifth order Markov model and functional sequence sites modeled as a position weight matrix (Geneid version 3).  Fully automatic, very fast.  Trained on Drosophila data

129 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.)  References  Guigó et al. (1998), JCB, 5,  URL  Information on training process: 

130 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.) Mark Borodovsky's Lab, School of Biology, Georgia Institute of Technology  Credit  Mark Borodovsky, John Besemer  Method  Markov chain models combined with HMM technology (Genemark.hmm).  URL 

131 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.) Biodivision, GSF Forschungszentrum für Umwelt und Gesundheit, Neuherberg, Germany  Credit  Matthias Scherf, Andreas Klingenhoff, Thomas Werner  Method  Universal sequence classifier which is based on a correlated word analysis to predict initiators and promoter associated TATA boxes (CoreInspector V1.0 beta). Sequences of 100 bp are classified at once.  Trained on Eukaryotic Promoter Database (EPD version 5.9).  Fully automatic, 2 seconds per 1Kb.  References  Scherf et al. (1999), in preparation.  URL 

132 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.) The Department of Biomathematical Sciences, Mount Sinai School of Medicine, New York  Credit  Gary Benson  Method  Tandem repeats finder (TRF v2.02) uses theoretical model of the similarity between adjacent copies of pattern (pattern from bp recognized); dynamic programming for candidate validation.  Fully automatic; very fast (seconds per 1Mb).   References  Benson (1999), Nucl. Acids Res., 27(2),  URL 

133 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.) Genie, UC Berkeley/UC Santa Cruz/ Neomorphic Inc.  Credit  Martin G. Reese, David Kulp, Hari Tammana, David Haussler  Method  Generalized hidden Markov model with optional integration of EST hits and homology searches (Genie).  Trained on Drosophila data  Semi-automatic, in that the overlaps of the analyzed sequence contigs (110kb) where manual run again with Genie to resolve conflicts.  BLAST used for homology searches on non-redundant protein database (nr).

134 Reese et al., Tutorial #3, ISMB ‘99 Submissions (cont.)  References  Reese et al. (1997), JCB, 4(3),  Kulp et al. (1997), Biocomputing: Proc. Of the 1997 PSB conference,  Kulp et al. (1996), ISMB, 4,  URL 

135 Reese et al., Tutorial #3, ISMB ‘99 Submission classes

136 Reese et al., Tutorial #3, ISMB ‘99 Submission classes (cont.)

137 Reese et al., Tutorial #3, ISMB ‘99 Gene finding techniques

138 Reese et al., Tutorial #3, ISMB ‘99 Measuring success By nucleotide  Sensitivity/Specificity (Sn/Sp) By exon  Sn/Sp  Missed exons (ME), wrong exons (WE) By gene  Sn/Sp  Missed genes (MG), wrong genes (WG)  Average overlap statistics Based on Burset and Guigo (1996), “Evaluation of gene structure prediction programs”. Genomics, 34(3),

139 Reese et al., Tutorial #3, ISMB ‘99 Definitions and formulae TP = True positive FP = False positive FN = False negative Sn = TP/(TP+FN) Sp = TP/(TP+FP)

140 Reese et al., Tutorial #3, ISMB ‘99 Genes: True positives (TP)

141 Reese et al., Tutorial #3, ISMB ‘99 Genes: False positives (FP)

142 Reese et al., Tutorial #3, ISMB ‘99 Genes: False Negatives (FN)

143 Reese et al., Tutorial #3, ISMB ‘99 Toy example 1 (1) Sn = TP/(TP+FN) Sp = TP/(TP+FP)

144 Reese et al., Tutorial #3, ISMB ‘99 Genes: Missing Genes (MG)

145 Reese et al., Tutorial #3, ISMB ‘99 Genes: Wrong Genes (WG)

146 Reese et al., Tutorial #3, ISMB ‘99 Toy example 1 (2) Sn = TP/(TP+FN) Sp = TP/(TP+FP)

147 Reese et al., Tutorial #3, ISMB ‘99 Genes: Std 1 versus Std 3 Std1: “conservative gene set” Std3: “more complete gene set”

148 Reese et al., Tutorial #3, ISMB ‘99 Toy example 1 (3) Sn = TP/(TP+FN) Sp = TP/(TP+FP)

149 Reese et al., Tutorial #3, ISMB ‘99 Genes: Std1 and Std3 versus “real” gene structure

150 Reese et al., Tutorial #3, ISMB ‘99 Toy example 1 (4)

151 Reese et al., Tutorial #3, ISMB ‘99 Toy example 1 (5): Exon level

152 Reese et al., Tutorial #3, ISMB ‘99 Genes: Joined genes (JG)

153 Reese et al., Tutorial #3, ISMB ‘99 Genes: Split genes (SG)

154 Reese et al., Tutorial #3, ISMB ‘99 Definition: “Joined” and “split” genes JG > 1, tendency to join multiple actual genes into one prediction SG > 1, tendency to split actual genes into separate gene predictions Inspired by Hayes and Guigó (1999), unpublished. SG = # Predicted genes that overlap actual genes # Actual genes that overlap one or more predicted genes JG = # Actual genes that overlap predicted genes # Predicted genes that overlap one or more actual genes

155 Reese et al., Tutorial #3, ISMB ‘99 Toy example 2 (1)

156 Reese et al., Tutorial #3, ISMB ‘99 Annotation experiment results Results available during tutorial and at

157 Reese et al., Tutorial #3, ISMB ‘99 Results: Base level Sensitivity:  Low variability among predictors  ~95% coverage of the proteome Specificity  ~90%  Programs that are more like Genscan (used for original annotation) might do better?

158 Reese et al., Tutorial #3, ISMB ‘99 Results: Exon level Higher variability among predictors Up to ~75% sensitivity (both exon boundaries correct) 55% specificity Low specificity because partial exon overlaps do not count Missing exons below 5% Many wrong exons (~20%)

159 Reese et al., Tutorial #3, ISMB ‘99 Results: Gene level

160 Reese et al., Tutorial #3, ISMB ‘99 Results: Gene level 60% of actual genes predicted completely correct Specificity only 30-40% 5-10% missed genes (comparable to Sanger Center) 40% wrong genes, a lot of short genes over-predicted (possibly not annotated in Standard 3) Splitting genes is a bigger problem than joining genes

161 Reese et al., Tutorial #3, ISMB ‘99 Results (protein homology): Base level

162 Reese et al., Tutorial #3, ISMB ‘99 Results (protein homology): Exon level

163 Reese et al., Tutorial #3, ISMB ‘99 Results (protein homology): Gene level

164 Reese et al., Tutorial #3, ISMB ‘99 Transcription Start Site (TSS): Standard 1

165 Reese et al., Tutorial #3, ISMB ‘99 TSS: Standard 3

166 Reese et al., Tutorial #3, ISMB ‘99 Results: TSS recognition

167 Reese et al., Tutorial #3, ISMB ‘99 Interesting gene examples: bubblegum

168 Reese et al., Tutorial #3, ISMB ‘99 Adh/Adhr (Alcohol dehydrogenase/Adh related)

169 Reese et al., Tutorial #3, ISMB ‘99 Adh/Adhr (cont..)

170 Reese et al., Tutorial #3, ISMB ‘99 osp (outspread) Contains Adh and Adhr embedded in an intron

171 Reese et al., Tutorial #3, ISMB ‘99 cact (cactus)

172 Reese et al., Tutorial #3, ISMB ‘99 kuz (kuzbanian)

173 Reese et al., Tutorial #3, ISMB ‘99 beat (beaten path)

174 Reese et al., Tutorial #3, ISMB ‘99 Idfg1, Idfg2, Idfg3 (Imaginal Disc Growth Factor)

175 Reese et al., Tutorial #3, ISMB ‘99 Idfg1, Idfg2, Idfg3 (cont.) Chitinase-related Gene function has changed (now a growth factor)

176 Reese et al., Tutorial #3, ISMB ‘99 Conclusion of GASP1 95% coverage of the proteome Base level prediction is easier, exon level prediction is harder Small genes over predicted (?) Long introns The high number of “wrong genes” indicates possible incomplete annotation in Standard 3 (Are there more genes?) HMM seems to currently be the best approach Major improvements in multiple gene regions

177 Reese et al., Tutorial #3, ISMB ‘99 Conclusion GASP1 (cont.) Much lower false positive rates Methods optimized for organism of interest do better Gene finding including homology not always improves prediction Split genes is more of a problem than joined genes No program is perfect

178 Reese et al., Tutorial #3, ISMB ‘99 Discussion GASP1 Genes in introns Alternative splicing Genomic contamination in cDNA libraries Translation start prediction Biological verification of prediction needed  Improve test bed by cDNA sequencing  More regulation data needed to confirm promoter assessment Combining methods Better methods needed GASP 2 ?

179 Reese et al., Tutorial #3, ISMB ‘99 Conclusions on annotating complete eukaryotic genomes Throughput has to improve dramatically Not only genes but also their relationships have to be elucidated Complete transcript cDNAs very powerful tool for annotation including alternative transcripts Comparative genomics as well as expression analysis improves/completes genome annotation Standardization efforts needed (ontology working group, OMG, OiB, NCBI/EBI, Bioxml, etc.)  Standards for description of gene products  Exchange format (GFF, Genbank, EMBL, XML)

180 Reese et al., Tutorial #3, ISMB ‘99 Conclusions on annotating complete eukaryotic genomes (cont.) Maintenance requires even more effort than the original development Automated methods are not good enough Human curators can cause problems too Functional assignment by homology is sometimes unreliable

181 Reese et al., Tutorial #3, ISMB ‘99 Discussion on annotating complete eukaryotic genomes Re-annotation: updating results and annotations over time  Genomic sequence changes (indels, point mutations)  Analysis software changes  New entries in public sequence databases  Entries removed from sequence databases Audit trail for annotations Master copy of genome annotations should reside in the model organism databases where the expertise resides Community collaborative annotation

182 Reese et al., Tutorial #3, ISMB ‘99 Acknowledgments Uwe Ohler (University of Erlangen, Germany) Gerry Rubin (UC Berkeley) Sima Misra (UC Berkeley) Erwin Frise (UC Berkeley) Roderic Guigó (Barcelona) GFF team (headed by Richard Bruskiewich, Sanger Centre) Assessment team: Michael Ashburner (EBI), Peer Bork (EMBL), Richard Durbin (Sanger), Roderic Guigó (Barcelona), Tim Hubbard (Sanger) Annotation experiment participants


Download ppt "Reese et al., Tutorial #3, ISMB ‘99 The challenge of annotating a complete eukaryotic genome: A case study in Drosophila melanogaster Martin G. Reese"

Similar presentations


Ads by Google