Presentation is loading. Please wait.

Presentation is loading. Please wait.

© 2014 SRI International Metabolomics Applications of the BioCyc Databases and Pathway Tools Software Peter D. Karpecocyc.org SRI Internationalbiocyc.org.

Similar presentations


Presentation on theme: "© 2014 SRI International Metabolomics Applications of the BioCyc Databases and Pathway Tools Software Peter D. Karpecocyc.org SRI Internationalbiocyc.org."— Presentation transcript:

1 © 2014 SRI International Metabolomics Applications of the BioCyc Databases and Pathway Tools Software Peter D. Karpecocyc.org SRI Internationalbiocyc.org metacyc.org

2 © 2014 SRI International Overview Overview of MetaCyc family of Pathway/Genome Databases (PGDBs) – BioCyc collection: EcoCyc, MetaCyc, HumanCyc, etc – Curated PGDBs for Arabidopsis, Yeast, Mouse, Fly, etc Overview of Pathway Tools software Automatic generation of metabolic-flux models

3 © 2014 SRI International Leverage Genome Information to Accelerate Metabolic Engineering Comprehensive databases on known metabolic pathways and enzymes Software to track and analyze the genome, metabolic network, and regulatory network of engineered organism Accelerate generation of steady-state metabolic flux models

4 © 2014 SRI International MetaCyc Family of Pathway/Genome Databases 6,000+ databases from many institutions All domains of life with microbial emphasis Genomes plus predicted metabolic pathways DBs derived from MetaCyc via computational pathway prediction Common schema Common controlled vocabularies Managed using Pathway Tools software Archives of Toxicology 85:1015 2011 BioCyc.org 3,500 MetaCyc Family 6,000+

5 © 2014 SRI International Curated Databases Within the MetaCyc Family DatabaseOrganismOrganizationPublications Curated From MetaCycMultiorganismSRI40,000 EcoCycE. coliSRI25,000 HumanCycH. sapiensSRI AraCycA. thalianaTAIR/Carnegie Institution 2,282 YeastCycS. cerevisiaeSGD/Stanford/SRI565 MouseCycM. musculusMGD/Jackson Laboratory http://biocyc.org/otherpgdbs.shtml

6 © 2014 SRI International Pathway Tools Software Comprehensive systems biology software environment Create and maintain an organism database integrating genome, pathway, regulatory information – Computational inference tools – Interactive editing tools Query and visualize that database Generate steady-state metabolic flux models – Flux-balance analysis Interpret omics datasets Comparative analysis tools Licensed by 5,000+ groups

7 © 2014 SRI International Motivations: Management of Metabolic Pathway Data Organize growing corpus of data on metabolic pathways – Experimentally elucidated pathways in the biomedical literature – Computationally predicted pathways derived from genome data Provide software tools for querying and comprehending this complex information space Multiorganism view: MetaCyc – Unique, experimentally elucidated pathways across all organisms – Reference database for computational pathway prediction Organism-specific view: – Organism-specific Pathway/Genome Databases – Detailed qualitative models of metabolic networks – Combine computational predictions with experimentally determined pathways

8 © 2014 SRI International Model Organism Databases / Organism Specific Databases DBs that describe the genome and other information about an organism Every sequenced organism with an active experimental community requires a MOD – Integrate genome data with information about the biochemical and genetic network of the organism – Integrate literature-based information with computational predictions Accurate metabolic modeling requires a curation effort

9 © 2014 SRI International Rationale for MODs Each “complete” genome is incomplete in several respects: – 40%-60% of genes have no assigned function – Roughly 7% of those assigned functions are incorrect – Many assigned functions are non-specific Need continuous updating of annotations with respect to new experimental data and computational predictions – Gene positions, sequence, gene functions, regulatory sites, pathways MODs are platforms for global analyses of an organism – Interpret omics data in a pathway context – In silico prediction of essential genes – Characterize systems properties of metabolic and genetic networks

10 © 2014 SRI International Pathway/Genome Database Chromosomes Plasmids Genes Proteins RNAs Reactions Pathways Compounds CELL Regulation Operons Promoters DNA Binding Sites Regulatory Interactions Sequence Features

11 © 2014 SRI International BioCyc Collection of 3,000 Pathway/Genome Databases Pathway/Genome Database (PGDB) – combines information about – Pathways, reactions, substrates – Enzymes, transporters – Genes, replicons – Transcription factors/sites, promoters, operons Tier 1: Literature-Derived PGDBs – MetaCyc, HumanCyc, YeastCyc – EcoCyc -- Escherichia coli K-12 – AraCyc – Arabidopsis thaliana Tier 2: Computationally-derived DBs, Some Curation -- 34 PGDBs – Bacillus subtilis, Mycobacterium tuberculosis Tier 3: Computationally-derived DBs, No Curation -- ~3,000 PGDBs

12 © 2014 SRI International What is Curation? Ongoing manual updating and refinement of a PGDB Correcting false-positive and false-negative predictions Incorporating information from experimental literature Authoring of comments and citations Updating database fields Gene positions, names, synonyms Protein functions, activators, inhibitors Addition of new pathways, modification of existing pathways Defining TF binding sites, promoters, regulation of transcription initiation and other processes

13 © 2014 SRI International BioCyc Tier 3 Tier 3 DBs created by subjecting annotated genomes to PathoLogic computational processing pipeline: – Predicts metabolic pathways – Predicts which genes code for missing enzymes in metabolic pathways – Infers transport reactions from transporter names – Predicts operons (bacteria) Tier 3 genomes are regenerated on a regular basis from RefSeq

14 © 2014 SRI International Obtaining a PGDB for Organism of Interest Find existing PGDB in BioCyc Find existing PGDB from larger MetaCyc family of PGDBs – http://biocyc.org/otherpgdbs.shtml http://biocyc.org/otherpgdbs.shtml Download from PGDB registry – http://biocyc.org/registry.html http://biocyc.org/registry.html Create your own PGDB

15 © 2014 SRI International Pathway Tools Software: PGDBs Created Outside SRI 4,000+ licensees: 250 groups applying software to 1,700 organisms Saccharomyces cerevisiae, SGD project, Stanford University – 135 pathways / 565 publications – BioCyc.org FungiCyc, Broad Institute Candida albicans, CGD project, Stanford University dictyBase, Northwestern University Mouse, MGD, Jackson Laboratory -- BioCyc.org Drosophila, FlyBase, Harvard University -- BioCyc.org Under development: – C. elegans, WormBase Arabidopsis thaliana, TAIR, Carnegie Institution of Washington – 288 pathways / 2282 publications – BioCyc.org PlantCyc: Poplar, Cassava, Corn, Grape, Soy, Carnegie Institution Six Solanaceae species, Cornell University GrameneDB: Rice, Sorghum, Maize, Cold Spring Harbor Laboratory Medicago truncatula, Samuel Roberts Noble Foundation ChlamyCyc, GoFORSYS

16 © 2014 SRI International Pathway Tools Software: PGDBs Created Outside SRI M. Bibb, John Innes Centre, Streptomyces coelicolor F. Brinkman, Simon Fraser Univ, Pseudomonas aeruginosa Genoscope, Acinetobacter R.J.S. Baerends, University of Groningen, Lactococcus lactis IL1403, Lactococcus lactis MG1363, Streptococcus pneumoniae TIGR4, Bacillus subtilis 168, Bacillus cereus ATCC14579 Matthew Berriman, Sanger Centre, Trypanosoma brucei, Leishmania major Sergio Encarnacion, UNAM, Sinorhizobium meliloti Mark van der Giezen, University of London, Entamoeba histolytica, Giardia intestinalis

17 © 2014 SRI International Pathway Tools Software: PGDBs Created Outside SRI Large scale users: – C. Medigue, Genoscope, 500+ PGDBs – J. Zucker, Broad Inst, 94 PGDBs – G. Sutton, J. Craig Venter Institute, 80+ PGDBs – G. Burger, U Montreal, 60+ PGDBs – E. Uberbacher, ORNL 33 Bioenergy-related organisms – Bart Weimer, UC Davis, Lactococcus lactis, Brevibacterium linens, Lactobacillus acidophilus, Lactobacillus plantarum, Lactobacillus johnsonii, Listeria monocytogenes Partial listing of outside PGDBs at http://biocyc.org/otherpgdbs.shtmlhttp://biocyc.org/otherpgdbs.shtml

18 © 2014 SRI International EcoCyc Project – EcoCyc.org E. coli Encyclopedia – Review-level Model-Organism Database for E. coli – Derived from 25,000 publications “Multi-dimensional annotation of the E. coli K-12 genome” – Gene product summaries and literature citations – Evidence codes – Gene Ontology terms – Protein features (active sites, metal ion binding sites) – Multimeric complexes – Metabolic pathways – Regulation of gene expression and of protein activity – Gene essentiality data – Growth under alternative nutrient conditions Nuc. Acids Res. 41:D605 2013 Karp, Gunsalus, Collado-Vides, Paulsen

19 © 2014 SRI International EcoCyc = E.coli Dataset + Pathway/Genome Navigator Genes: 4,499 Monomers: 4389 Complexes: 976 RNAs: 301 Reactions: Metabolic: 1600 Transport: 370 Pathways: 312 Compounds: 2,400 Regulation: Operons: 4,500 Trans Factors: 222 Promoters: 3,770 TF Binding Sites: 2,700 Reg Interactions: 5,900 EcoCyc v17.0 Citations: 24,000 URL: EcoCyc.org

20 © 2014 SRI International Perspective 1: EcoCyc as Online Encyclopedia All gene products for which experimental literature exists are curated with a minireview summary – 3,730 gene products contain summaries – Summaries cover function, interactions, mutant phenotypes, crystal structures, regulation, and more Additional summaries and other data found in pages for genes, operons, pathways Quick Search

21 © 2014 SRI International Perspective 2: EcoCyc as Queryable Database High-fidelity knowledge representation amenable to structured queries 333 database fields capture object properties and relationships Each molecular species defined as a DB object – Genes, proteins, small molecules Each molecular interaction defined as a DB object – Metabolic and transport reactions, regulation Extensive search tools – Object-specific search Search Menu – Advanced search Search -> Advanced

22 © 2014 SRI International Paradigm 3: EcoCyc as Predictive Metabolic Model A steady-state quantitative model of E. coli metabolism can be generated from EcoCyc Predicts phenotypes of E. coli knock-outs, and growth/no-growth of E. coli on different nutrients Model is updated on each EcoCyc release Serves as a quality check on the EcoCyc data

23 © 2014 SRI International EcoCyc Procedures DB updates performed by 5 staff curators – Information gathered from biomedical literature Enter data into structured database fields Author extensive summaries Update evidence codes – Corrections submitted by E. coli researchers Three releases per year Quality assurance of data and software – Evaluate database consistency constraints – Perform element balancing of reactions – Run other checking programs

24 © 2014 SRI International EcoCyc Accelerates Science Experimentalists – E. coli experimentalists – Experimentalists working with other microbes – Analysis of expression data Computational biologists – Biological research using computational methods – Genome annotation – Study properties of E. coli metabolic and regulatory networks Bioinformaticists – Training and validation of new bioinformatics algorithms – predict operons, promoters, protein functional linkages, protein-protein interactions, Metabolic engineers – “Design of organisms for the production of organic acids, amino acids, ethanol, hydrogen, and solvents “ Educators – Microbiology and metabolism education

25 © 2014 SRI International Recent Developments in EcoCyc EcoCyc contains six knock-out datasets for E. coli containing 13,000 growth observations ReferenceMediumNo GrowthGrowthIndeterminate Gerdes03 LB enriched, aerobic 61430822 Baba06 LB Lennox, aerobic 30039061 Baba06 MOPS+0.4% glucose, aerobic 460482392 Feist07 MOPS+0.4% glucose, aerobic 460482392 Patrick07 M9+0.4% glucose, aerobic 107 Joyce06 M9+1% glucose, aerobic 11837631

26 © 2014 SRI International Recent Developments in EcoCyc – Growth-Observation Data EcoCyc contains 1831 growth observations under 522 conditions for E. coli Substantial number of discrepancies – 45 cases remain where growth status is unclear AerobicAnaerobic Low throughput data from literature 230 Low throughput data from our group 200 PM data from literature1244191 PM data from our group3530

27 © 2014 SRI International

28 MetaCyc : Metabolic Encyclopedia Describes experimentally determined metabolic pathways, reactions, enzymes, and compounds Literature-based DB with extensive references and commentary MetaCyc vs BioCyc: Experimentally elucidated pathways Jointly developed by – P. Karp, R. Caspi, C. Fulcher, SRI International – L. Mueller, A. Pujar, Boyce Thompson Institute – S. Rhee, P. Zhang, Carnegie Institution Nucleic Acids Research 2012 Database Issue

29 © 2014 SRI International Applications of MetaCyc Reference source on metabolic pathways Predict pathways from genomes Metabolic engineering – Find enzymes with desired activities, regulatory properties – Determine cofactor requirements Systematic studies of metabolism Computer-aided education

30 © 2014 SRI International MetaCyc Data -- Version 18.0 Pathways 2,200 Reactions 11,700 Enzymes 9,700 Small Molecules 11,000 Organisms2,500 Citations 40,600 “A Systematic Comparison of the MetaCyc and KEGG Pathway Databases BMC Bioinformatics 2013 14(1):112

31 © 2014 SRI International Taxonomic Distribution of MetaCyc Pathways Version 17.5 Bacteria1,130 Green Plants 830 Fungi 300 Metazoa 275 Archaea148

32 © 2014 SRI International Biosynthesis [902] – Amino acids Biosynthesis [105] – Aromatic Compounds Biosynthesis [13] – Carbohydrates Biosynthesis [70] – Cell structures Biosynthesis [31] – Cofactors, Prosthetic Groups, Electron Carriers Biosynthesis [160] – Hormones Biosynthesis [40] – Fatty Acids and Lipids Biosynthesis [101] – Metabolic Regulators Biosynthesis [4] – Nucleosides and Nucleotides Biosynthesis [20] – Amines and Polyamines Biosynthesis [32] – Secondary Metabolites Biosynthesis [351] Antibiotic Biosynthesis [20] Fatty Acid Derivatives Biosynthesis [7] Flavonoids Biosynthesis [70] Nitrogen-Containing Secondary Compounds Biosynthesis [64] – Alkaloids Biosynthesis [43] Phenylpropanoid Derivatives Biosynthesis [46] Phytoalexins Biosynthesis [25] Sugar Derivatives Biosynthesis [10] Terpenoids Biosynthesis [103] – Siderophore Biosynthesis [7]

33 © 2014 SRI International Degradation/Utilization/Assimilation [639] – Alcohols Degradation [14] – Aldehyde Degradation [12] – Amines and Polyamines Degradation [40] – Amino Acids Degradation [113] – Aromatic Compounds Degradation [152] – C1 Compounds Utilization and Assimilation [24] – Carbohydrates Degradation [52] – Carboxylates Degradation [30] – Chlorinated Compounds Degradation [39] – Cofactors, Prosthetic Groups, Electron Carriers Degradation [2] – Fatty Acid and Lipids Degradation [18] – Inorganic Nutrients Metabolism [72] Nitrogen Compounds Metabolism [15] Phosphorus Compounds Metabolism [3] Sulfur Compounds Metabolism [54] – Nucleosides and Nucleotides Degradation and Recycling [9] – Secondary Metabolites Degradation [58] Nitrogen Containing Secondary Compounds Degradation [13] Sugar Derivatives Degradation [31] Terpenoids Degradation [10]

34 © 2014 SRI International Generation of precursor metabolites and energy [124] – Chemoautotrophic Energy Metabolism [14] Hydrogen Oxidation [2] – Electron Transfer [11] – Fermentation [34] – Glycolysis [6] – Methanogenesis [12] – Pentose Phosphate Pathways [4] – Photosynthesis [6] – Respiration [25] Aerobic Respiration [9] Anaerobic Respiration [14] – TCA cycle [9]

35 © 2014 SRI International Detoxification [16] – Acid Resistance [2] – Arsenate Detoxification [3] – Mercury Detoxification [1] – Methylglyoxal Detoxification [8]

36 © 2014 SRI International Pathway Boundaries A connected sequence of biochemical reactions Occurs in one organism Conserved through evolution Regulated as a unit Starts or stops at one of 13 common intermediate metabolites – glucose-6-phosphate, fructose-6-phosphate – ribose-5-phosphate, erythrose-4-phosphate, triose phosphate – 3-phosphoglycerate, phosphoenolpyruvate – pyruvate, acetyl CoA, alpha-oxoglutarate, succinyl CoA – oxaloacetate, sedoheptulose-7-phosphate

37 © 2014 SRI International MetaCyc Pathway Variants Pathways that accomplish similar biochemical functions using different biochemical routes – Alanine biosynthesis I – E. coli – Alanine biosynthesis II – H. sapiens Pathways that accomplish similar biochemical functions using similar sets of reactions – Several variants of TCA Cycle

38 © 2014 SRI International MetaCyc Super-Pathways Groups of pathways linked by common substrates Example: Super-pathway containing – Chorismate biosynthesis – Tryptophan biosynthesis – Phenylalanine biosynthesis – Tyrosine biosynthesis Super-pathways defined by listing their component pathways Multiple levels of super-pathways can be defined Pathway layout algorithms accommodate super-pathways

39 © 2014 SRI International Enzyme Data Available in MetaCyc Reaction(s) catalyzed Alternative substrates Activators, inhibitors, cofactors, prosthetic groups Subunit structure Genes Features on protein sequence Cellular location pI, molecular weight, Km, Vmax Gene Ontology terms Links to other bioinformatics databases

40 © 2014 SRI International MetaCyc Curation DB updates by 5 staff curators – Information gathered from biomedical literature – Emphasis on microbial and plant pathways – More prevalent pathways given higher priority Review-level database Four releases per year Quality assurance of data and software: – Evaluate database consistency constraints – Perform element balancing of reactions – Run other checking programs – Display every DB object

41 © 2014 SRI International Comparison with KEGG KEGG vs MetaCyc: Reference pathway collections – KEGG maps are not pathways Nuc Acids Res 34:3687 2006 KEGG maps contain multiple biological pathways KEGG maps are composites of pathways in many organisms -- do not identify what specific pathways elucidated in what organisms Two genes chosen at random from a BioCyc pathway are more likely to be related according to genome context methods than from a KEGG pathway – KEGG has few literature citations, few comments, less enzyme detail KEGG vs organism-specific PGDBs – KEGG does not curate or customize pathway networks for each organism – Highly curated PGDBs now exist for important organisms such as E. coli, yeast, mouse, Arabidopsis

42 © 2014 SRI International Comparison of Pathway Tools to KEGG Inference tools – KEGG does not predict presence or absence of pathways – KEGG lacks pathway hole filler, operon predictor Curation tools – KEGG does not distribute curation tools – No ability to customize pathways to the organism – Pathway Tools schema much more comprehensive Visualization and analysis – KEGG does not perform automatic pathway layout – KEGG metabolic-map diagram extremely limited – No comparative pathway analysis

43 © 2014 SRI International EcoCyc and MetaCyc Review level databases Data derived primarily from biomedical literature – Manual entry by staff curators – Updates by staff curators only DBMS: Frame knowledge representation system Data validation – Consistency constraints – Lisp programs that verify other semantic relationships Unbalanced chemical reactions

44 44 SRI International Bioinformatics Pathway Tools

45 © 2014 SRI International Pathway Tools Software PathoLogic – Predicts operons, metabolic network, pathway hole fillers, from genome – Computational creation of new Pathway/Genome Databases Pathway/Genome Editors – Distributed curation of PGDBs – Distributed object database system, interactive editing tools Pathway/Genome Navigator – WWW publishing of PGDBs – Querying, visualization of pathways, chromosomes, operons – Analysis operations Pathway visualization of gene-expression data Global comparisons of metabolic networks Briefings in Bioinformatics 11:40-79 2010

46 © 2014 SRI International Pathway Tools Software Pathway/Genome Editors Pathway/Genome Database PathoLogic MetaCyc Annotated Genome Pathway/Genome Navigator Briefings in Bioinformatics 11:40-79 2010 + MetaFlux

47 © 2014 SRI International Pathway Tools Enables Multi-Use Metabolic Databases Zoomable Metabolic Map Queryable Database Omics Data Analysis Metabolic Model Encyclopedia

48 © 2014 SRI International Pathway Tools Software: PathoLogic Computational creation of new Pathway/Genome Databases Transforms genome into Pathway Tools schema and layers inferred information above the genome Predicts operons Predicts metabolic network Predicts which genes code for missing enzymes in metabolic pathways Infers transport reactions from transporter names

49 © 2014 SRI International Pathway Tools Software: Pathway/Genome Editors Interactively update PGDBs with graphical editors Support geographically distributed teams of curators with object database system Gene and protein editor Reaction editor Compound editor Pathway editor Operon editor Publication editor

50 © 2014 SRI International Pathway Tools Software: Pathway/Genome Navigator Querying and visualization of: – Pathways – Reactions – Metabolites – Genes/Proteins/RNA – Regulatory interactions – Chromosomes Modes of operation: – Web mode – Desktop mode – Most functionality shared

51 © 2014 SRI International Pathway Tools Software: MetaFlux Speeds development of genome-scale metabolic flux models Steady-state quantitative flux-models generated directly from PGDBs Computed reaction fluxes can be painted onto metabolic overview diagram Multiple gap filler accelerates model development by suggesting model completions: – Reactions to add from MetaCyc – Additional nutrients and secreted compounds

52 © 2014 SRI International Pathway Tools Schema / Ontology 1064 classes – Datatype classes such as: Pathways, Reactions, Compounds, Macromolecules, Proteins, Replicons, DNA-Segments (Genes, Operons, Promoters) – Taxonomies for Pathways, Reactions, Compounds – Cell Component Ontology – Evidence Ontology 308 attributes and relationships Span genome, metabolism, regulatory information – Meta-data: Creator, Creation-Date – Comment, Citations, Common-Name, Synonyms – Attributes: Molecular-Weight, DNA-Footprint-Size – Relationships: Catalyzes, Component-Of, Product

53 © 2014 SRI International Pathway Tools Architecture Ocelot DBMS GFP API Pathway Genome Navigator Web Mode Desktop Mode Protein Editor Pathway Editor Reaction Editor Oracle or MySQL Disk File Lisp Perl Java

54 © 2014 SRI International Ocelot Storage System Architecture Persistent storage via disk files or Oracle or MySQL – Concurrent development: Oracle or MySQL – Single-user development: disk files Oracle/MySQL DBMS storage – DBMS is submerged within Ocelot, invisible to users – Frames transferred from DBMS to Ocelot On demand By background prefetcher Frames are cached in memory Persistent disk cache to speed performance via Internet Transaction logging facility

55 © 2014 SRI International Ocelot Knowledge Server Schema Evolution Frame data model – Minimizes size of schema relative to semantic complexity Schema is stored within the DB Schema is self documenting Schema evolution facilitated by – Easy addition/removal of slots, or alteration of slot datatypes – Flexible data formats that do not require dumping/reloading of data

56 © 2014 SRI International PathoLogic Processing 1.Translate source genome to PGDB form 2.Predict operons 3.Predict metabolic pathways 4.Predict pathway hole fillers 5.Transport inference parser 6.Build metabolic overview diagram

57 © 2014 SRI International PathoLogic Step 1: Translate Genome to PGDB Pathway/Genome Database Annotated Genomic Sequence Genes/ORFs Gene Products DNA Sequences Reactions Pathways Compounds Multi-organism Pathway Database (MetaCyc) PathoLogic Software Integrates genome and pathway data to identify putative metabolic networks Genomic Map Genes Gene Products Reactions Pathways Compounds

58 © 2014 SRI International

59 PathoLogic Step 2: Predict Operons Predict adjacent genes A and B in same operon based on: – Intragenic distance – Functional relatedness of A and B Tests for functional relatedness: – A and B in same gene functional class (MultiFun) – A and B in same metabolic pathway – A codes for enzyme in a pathway and B codes for transporter involving a substrate in that pathway – A and B are monomers in same protein complex Correctly predicts 80% of E. coli transcription units Marks predicted operons with computational evidence codes Bioinformatics 20:709-17 2004

60 © 2014 SRI International PathoLogic Step 3: Prediction of Metabolic Pathways Use the set of reactions catalyzed by the genome to infer what metabolic pathways operate in a cell Two steps in metabolic pathway inference: – Inference of the reactome – Inference of metabolic pathways by selecting from reference pathways Dale et al, BMC Bioinformatics 2010 11:15

61 © 2014 SRI International Pathway Prediction Pathway prediction is useful because – Pathways organize the metabolic network into mentally tractable units – Pathways guide us to search for missing enzymes – Pathway inference fills in holes in the metabolic network – Pathways can be used for analysis of high-throughput data Visualization, enrichment analysis Pathway prediction is hard because – Reactome inference is imperfect – Some reactions present in multiple pathways – Pathway variants share many reactions in common – Increasing size of MetaCyc

62 © 2014 SRI International Reactome Inference For each protein in the organism, infer reaction(s) it catalyzes Protein functions can be specified in three ways: – Enzyme names (protein functions) (uncontrolled vocabulary) – EC numbers – Gene Ontology terms Detect conflicts among this information – Example: Yersinia pseudotuberculosis PB1 2-succinyl-5-enolpyruvyl-6-hydroxy-3-cyclohexene-1-carboxylate synthase / EC 4.1.1.71

63 © 2014 SRI International Enzyme Name Matching Extraneous information found in gene product names Putative carbamate kinase, alpha subunit Carbamate kinase (abcD) Carbamate kinase (3.2.1.4) Monoamine oxidase B bifunctional proline dehydrogenase/pyrroline-5-carboxylate dehydrogenase

64 © 2014 SRI International Reactome Inference -- Caveats Many genomes do not contain EC numbers – 176 of the 723 microbial genomes in CMR contain < 50 EC#s EC system is incomplete – MetaCyc database (NAR 2010) describes 8409 metabolic reactions – 3173 of those reactions lack an EC number EC system is ambiguous – 596 cases where an EC# refers to more than 1 reaction – 47 cases where an EC# refers to more than 5 reactions – One EC# refers to 39 reactions!

65 © 2014 SRI International Reactome Inference -- Caveats Most genomes do not contain GO terms GO has molecular function terms for enzymes that catalyze MetaCyc reactions – ftp://ftp.geneontology.org/pub/go/external2go/metacyc2go Key need: Annotation pipelines that infer GO terms for enzyme molecular functions

66 © 2014 SRI International Inference of Metabolic Pathways For each pathway in MetaCyc consider – What fraction of its reactions are present in the just-inferred reactome of the organism? – Are enzymes present for reactions unique to the pathway? – Are enzymes present for designated “key reactions” within MetaCyc pathways? Calvin cycle / ribulose bisphosphate carboxylase – Is a given pathway outside its designated taxonomic range? Calvin cycle: green plants, green algae, etc Standards in Genomic Sciences 5:424-429 2011

67 © 2014 SRI International Evaluation of Pathway Inference Define gold-standard pathway prediction set – E. coli, Yeast, Arabidopsis, Synechococcus, Mouse – Positive and negative pathways PathoLogic achieved 91% accuracy BMC Bioinformatics 11:15 2010

68 © 2014 SRI International Match Enzymes to Reactions Match Gene product 5.1.3.2 UDP-glucose-4- epimerase yes Assign no Probable enzyme -ase noyes Not a metabolic enzyme Manually search yes Assign no Can’t Assign MetaCyc UDP-D-glucose  UDP-galactose 2057 proteins matched by EC# 314 matched by name 1320 625

69 © 2014 SRI International Pathway Prediction Algorithm Two pathway lists: – U: Undecided status – K: Keep Initialize U to contain all MetaCyc pathways for which at least one reaction has an enzyme

70 © 2014 SRI International Pathway Prediction Algorithm For each P in U: – If current organism is outside taxonomic range of P AND at least one reaction in P lacks an enzyme, delete P from U – If all reactions of P designated as key reactions have no enzyme, delete P from U

71 © 2014 SRI International Pathway Prediction Algorithm Iterate through P in U until U is unchanged: – If P should be kept, move P to K A reaction in P is unique to P and has an enzyme At most one reaction in P has no enzyme The enzymes present for P are not a subset of the enzymes present for a variant pathway of P – If P should be deleted, delete P from U At most one reaction R in P has an enzyme, and R is not unique to P The pathway is a biosynthetic pathway missing its final steps The pathway is a catabolic pathway missing its initial steps Accuracy: 91%

72 © 2014 SRI International Pathway Prediction Prediction is hard because – Reactome inference is ambiguous – Some reactions present in multiple pathways – Pathway variants share many reactions in common

73 © 2014 SRI International Pathway Evidence Report

74 © 2014 SRI International Comparison with KEGG KEGG vs MetaCyc: Reference pathway collections – KEGG maps are not pathways Nuc Acids Res 34:3687 2006 KEGG maps contain multiple biological pathways KEGG maps are composites of pathways in many organisms -- do not identify what specific pathways elucidated in what organisms – KEGG modules are incomplete – KEGG has few literature citations, few comments, less enzyme detail KEGG vs organism-specific PGDBs – KEGG does not curate or customize pathway networks for each organism – Highly curated PGDBs now exist for important organisms such as E. coli, yeast, mouse, Arabidopsis KEGG algorithms – Not published; accuracy unknown

75 © 2014 SRI International Pathway Analysis of Metagenomes Bin the metagenome data and create separate PGDBs for each organism – Hallam lab Compute list of all pathways present in the metagenome

76 © 2014 SRI International Missing Enzymatic Functions Globally missing functions – Biochemically characterized enzymes that have never been sequenced – For 1407 (38%) enzymes with assigned EC#s, no sequence exists in public sequence DBs Genome Biology 5:401 2004 Locally missing functions – Holes in predicted metabolic pathways – Average of 280 holes in 3 microbial genomes

77 © 2014 SRI International PathoLogic Step 4: Pathway Hole Filler Definition: Pathway Holes are reactions in metabolic pathways for which no enzyme is identified holes L-aspartate iminoaspartate quinolinate nicotinate nucleotide deamido-NAD NAD 1.4.3.- 2.7.7.18 quinolinate synthetase nadA n.n. pyrophosphorylase nadC NAD+ synthetase, NH3 - dependent CC3619 6.3.5.1

78 © 2014 SRI International organism 1 enzyme A organism 2 enzyme A organism 5 enzyme A organism 4 enzyme A organism 3 enzyme A organism 8 enzyme A organism 7 enzyme A organism 6 enzyme A Step 1: Query UniProt for all sequences having EC# of pathway hole Step 2: BLAST against target genome Step 3 & 4: Consolidate hits and evaluate evidence 7 queries have high-scoring hits to sequence Y gene Y gene X gene Z

79 © 2014 SRI International Bayes Classifier P(protein has function X| E-value, avg. rank, aln. length, etc.) protein has function X % of query aligned avg. rank in BLAST output bestE-value Number of queries pwy directon adjacent rxns

80 © 2014 SRI International Pathway Hole Filler Why should hole filler find things beyond the original genome annotation? Reverse BLAST searches more sensitive Reverse BLAST searches find second domains Integration of multiple evidence types

81 © 2014 SRI International Caulobacter crescentus Pathway Holes 130 pathways containing 582 reactions 92 pathways contain 236 pathway holes Caulobacter holes filled: 77 holes filled at P >0.9 Previous functions of candidate hole fillers: – No predicted function – Correctly assigned single function – Incorrectly assigned function – Imprecise functional assignment BMC Bioinformatics 5:76 2004

82 © 2014 SRI International Example Pathway holes L-aspartate iminoaspartate quinolinate nicotinate nucleotide deamido-NAD NAD 1.4.3.- 2.7.7.18 quinolinate synthetase nadA (CC2912) n.n. pyrophosphorylase nadC (CC2915) NAD+ synthetase, NH3 - dependent CC3619 6.3.5.1 CC2913, P=0.99 CC3431*, P=0.90 CC3619, P=0.99 CC2913 L-aspartate oxidase (wrong EC# on rxn) CC3431 ORF CC3619 put. NAD(+)-synthetase (multidomain)

83 © 2014 SRI International PathoLogic Step 5: Transport Inference Parser Problem: Write a program to query a genome annotation to compute the substrates an organism can transport Typical genome annotations for transporters: – ATP transporter for ribose – ribose ABC transporter – D-ribose ATP transporter – ABC transporter, membrane spanning protein [ribose] – ABC transporter, membrane spanning protein [D-ribose]

84 © 2014 SRI International PathoLogic Step 5: Transport Inference Parser Input: “ATP transporter of phosphonate” Output: Structured description of transport activity Locates most transporters in genome annotation using keyword analysis Parse transporter names using a series of rules to identify: – Transported substrate, co-substrate – Influx/efflux – Energy coupling mechanism Creates transport reaction object: phosphonate [periplasm] + H 2 O + ATP = phosphonate + P i + ADP

85 © 2014 SRI International Transport Inference Parser Permits symbolic computation with transport activities: – Compute transportable substrates of the cell – Compute connectivity among compartments for substrates – Facilitate reasoning about transport/metabolism connections – Draw transport cartoon in protein pages, cellular overview

86 © 2014 SRI International Transport Inference Parser User reviews all assignments using interactive tool that allows assignments to be revised User also reviews transporters for which no assignment was made

87 © 2014 SRI International Analysis of High Throughput Datasets Generated automatically from PGDB Magnify, interrogate Omics viewers paint omics data onto overview diagrams – Different perspectives on same dataset – Use animation for multiple time points or conditions Genome-scale visualizations of cellular networks

88 © 2014 SRI International Cellular Overview Diagram Combines metabolic map and transporters Automatically generated, organism-specific Zoomable, queryable

89 © 2014 SRI International

90

91 E. coli Cellular Overview

92 © 2014 SRI International

93

94

95

96 Omics Data Graphing on Cellular Overview

97 © 2014 SRI International

98

99 Genome Overview

100 © 2014 SRI International Genome Poster

101 © 2014 SRI International Genome Overview

102 © 2014 SRI International Regulatory Overview Show regulatory relationships among gene groups

103 © 2014 SRI International Regulatory Omics Viewer

104 © 2014 SRI International Recent Developments Comprehensive treatment of bacterial regulation – Regulation of transcription initiation – Attenuation (6 types) – Regulation of translation by small RNAs and proteins – Substrate level regulation of enzyme activity – Protein modification Editing and drawing of signaling pathways

105 © 2014 SRI International Genome Browser ChIP-Chip Data Shown in Graph Track

106 © 2014 SRI International The Atom-Mapping Problem Definition: An atom-mapping is a bijection from reactant atoms to product atoms that specifies the terminus of each reaction atom MetaCyc v17.5 contains 10,300 atom mappings

107 © 2014 SRI International Applications of Atom Mapping Speed visual comprehension of reactions and pathways

108 © 2014 SRI International Applications of Atom Mapping Improve evaluation of computer-generated metabolic pathways – Do any feedstock atoms reach target compound? – What fraction of feedstock atoms reach target compound? Facilitate design and interpretation of isotope-labeling experiments

109 © 2014 SRI International Atom Mapping: Our Approach Weighted Minimal Bond-Edit Distance – Edit distance weighted by bond type and atom species – Computed using MILP for 9,390 MetaCyc reactions – Average time per reaction: 73% are solved in less than 1 second 96% are solved in less than 60 seconds – 96% of reactions had 1 or 2 solutions (with symmetries removed) – different bonds made/broken Solution times are a function of the solver – SCIP vs CPLEX J Chem Inf Model. 2012 52:2970-82.

110 © 2014 SRI International Accuracy of Our Atom Mappings Use KEGG RPAIR as a gold standard – Caveats: Not clear which RPAIRs are curated; accuracy of RPAIR unclear We implemented software to – Import KEGG and RPAIR into a Pathway Tools PGDB – Map atoms in KEGG reactions to corresponding atoms in MetaCyc reactions 2,446 atom mappings from KEGG RPAIR could be compared to MetaCyc mappings – 25 disagreements: – 1 reaction: experimental evidence our mapping is correct – 2 reactions: similar to preceding – 22 reactions – KEGG is correct

111 © 2014 SRI International RouteSearch Software --- Metabolism->Metabolic Route Search User specifies feedstock compound and target compound Software computes minimal-cost paths from feedstock to target based on reactions from – Current PGDB, plus, optionally – MetaCyc Optimality criteria: minimize – Number of reactions – Number of lost atoms based on atom mappings – Number of reactions foreign to the organism User interface guides exploration of solution pathways Latendresse et al., Bioinformatics 2014

112 © 2014 SRI International Results: Sample Metabolic Engineering Problems Five engineered pathways obtained from literature – 2-oxoisovalerate  3-methylbutanol (3-methylbutanol biosynthesis) – pyruvate  isopropanol (isopropanol biosynthesis) – pyruvate  n-butanol (pyruvate fermentation to butanol II) – 3-phospho-D-glycerate  n-butanol (1-butanol autotrophic biosynthesis) – 3-dehydroquinate  vanillin (vanillin biosynthesis) Given feedstock and target compounds, our system found the literature pathway in all five cases

113 © 2014 SRI International Pyruvate  Isopropanol Two highest-ranked pwys shown Best corresponds to pwy from literature Search engine can continue to generate alternatives

114 © 2014 SRI International Metabolomics Applications MetaCyc contains extensive multiorganism metabolite database Organism-specific metabolite databases in each PGDB – Genome+pathway context for interpreting metabolomics data Monoisotopic mass searches Paint metabolomics data onto pathway maps Group transformations Enrichment analysis

115 © 2014 SRI International

116 Object Groups Collect and save lists of metabolites, genes, pathways, … Share groups with colleagues

117 © 2014 SRI International Object Groups Create manually, from files, from query results Explore gene list interactively Combine (union, intersection, subtraction)

118 © 2014 SRI International Manipulate Genes and Sequences

119 © 2014 SRI International Object Group Transformations Transform metabolite group into group of metabolic pathways, then into gene group Create group containing transcriptional regulator; transform to all genes it regulates Transform gene group into group of regulators of those genes Transform gene group into list of TF binding sites controlling those genes; into list of sequences Create group of nucleotide positions; transform to closest genes; paint to cellular overview

120 © 2014 SRI International Object Groups: Enrichment Analysis “My experiments yielded a set of genes/metabolites. What do they have in common?” Given a group of genes: – What GO terms are statistically over-represented in that set? – What metabolic pathways are over-represented? – What transcriptional regulators are over-represented? Given a set of metabolites: – What metabolic pathways are statistically over-represented in that set?

121 © 2014 SRI International Symbolic Systems Biology Definition: Global analyses of biological systems using symbolic computing

122 © 2014 SRI International Symbolic Systems Biology “Symbolic computing is concerned with the representation and manipulation of information in symbolic form. It is often contrasted with numeric representation.” -- R. Cameron Examples of symbolic computation: – Symbolic algebra programs, e.g., Mathematica, Graphing Calculator – Compilers and interpreters for programming languages – Database query languages – Text analysis programs, e.g., Google – String matching for DNA and protein sequences – Artificial Intelligence methods, e.g., expert systems, symbolic logic, machine learning, natural language understanding

123 © 2014 SRI International Symbolic Systems Biology Concerned with different questions than quantitative systems biology Symbolic analyses can in many cases produce answers when quantitative approaches fail because of lack of parameters or intractable mathematics Symbolic computation is intimately dependent on the use of structured ontologies

124 © 2014 SRI International Pathway Tools Ontology 1064 classes – Main classes such as: Pathways, Reactions, Compounds, Macromolecules, Proteins, Replicons, DNA- Segments (Genes, Operons, Promoters) – Taxonomies for Pathways, Reactions, Compounds 308 slots – Meta-data: Creator, Creation-Date – Comment, Citations, Common-Name, Synonyms – Attributes: Molecular-Weight, DNA-Footprint-Size – Relationships: Catalyzes, Component-Of, Product Classes, instances, slots all stored side by side in DBMS

125 © 2014 SRI International Symbolic Computation on PGDBs: Complex Query Show metabolic enzymes regulated by a specified transcription factor For transcription factor F: Find all promoters F regulates – Find all genes in the operons controlled by those promoters Find their protein products – Find the reactions they catalyze » Highlight them in the diagram

126 © 2014 SRI International Critiquing the Parts List Slide thanks to Hirotada Mori (minus the banana!)

127 © 2014 SRI International Dead End Metabolite Finder A small molecule C is a dead-end if: – C is produced only by SMM reactions in Compartment, and no transporter acts on C in Compartment OR – C is consumed only by SMM reactions in Compartment, and no transporter acts on C in Compartment

128 128 SRI International Bioinformatics Automated Generation of Metabolic Flux Models from PGDBs Joint work with Mario Latendresse

129 © 2014 SRI International Marriage of Systems Biology and Model-Organism Databases Systems biology – Qualitative system-level analysis – Quantitative system-level modeling Hypothesis: Strong synergies between MODs and SB Curation is critical to SB and to MODs – Biological models undergo long periods of updating and refinement – Common curation effort for MOD and systems-biology model MOD provides data needed for SB construction and validation SB identifies errors and omissions in MOD, directs curation Methodologies from MODs can benefit systems-biology models – Evidence codes – Mini-review summaries – Literature citations

130 © 2014 SRI International Flux-Balance Analysis Nutrients Biomass Secretions A ABC X DD Metabolic Reaction List Steady state, constraint-based quantitative models of metabolism Starting information for organism of interest:

131 © 2014 SRI International Flux Balance Analysis Define system of linear equations encoding fluxes on each metabolite M – R1 + R2 = R3 + R4 + R5 Boundary reactions: – Exchange fluxes for nutrients and secretions – Biomass reaction L-arginine … + GTP … + …  biomass Submit to linear optimization package – Optimize biomass production – Optimize ATP production – Optimize production of desired end product M R1 R2 R4 R5 R3

132 © 2014 SRI International Example 100 40 O2O2 glucose2 pyruvate alanine ATP glucose O2O2 Biomass: ATP:alanine 4:1 160

133 © 2014 SRI International Pathway Prediction Pathway prediction is useful because – Pathways organize the metabolic network into mentally tractable units – Pathways guide us to search for missing enzymes – Pathway inference fills in holes in the metabolic network – Pathways can be used for analysis of high-throughput data Visualization, enrichment analysis Pathway prediction is hard because – Reactome inference is imperfect – Some reactions present in multiple pathways – Pathway variants share many reactions in common – Increasing size of MetaCyc

134 © 2014 SRI International FBA Results FBA predicts steady-state reaction fluxes for the metabolic network Remove reactions from model to predict knock-out phenotypes Supply alternative nutrient sets to predict growth phenotypes Predict growth rates, nutrient uptake rates

135 © 2014 SRI International Approach: Generate FBA Models from Pathway/Genome Databases Store and update metabolic model within PGDB – All query and visualization tools applicable to FBA model – FBA model is tightly coupled to genome and regulatory information MetaFlux generates linear programming problem from PGDB reactions Submit to constraint solver for model execution/solving Tools to accelerate model refinement: – Reaction balance checking – Dead-end metabolite analysis – Visualize reaction flux using cellular overview – Multiple gap filling MetaFlux: Latendresse et al, Bioinformatics 2012 28:388-96

136 © 2014 SRI International MetaFlux FBA Model Execution MetaFlux creates.lp file and executes SCIP solver – Konrad-Zuse-Zentrum für Informationstechnik Berlin Interpret SCIP output – Determine if SCIP found a solution – Map fluxes to PGDB reactions Display resulting fluxes on the Cellular Overview

137 © 2014 SRI International Model Debugging Via Dead End Metabolite Finder A small molecule C is a dead-end if: – C is produced only by metabolic reactions in Compartment, and no transporter acts on C in Compartment OR – C is consumed only by metabolic reactions in Compartment, and no transporter acts on C in Compartment

138 © 2014 SRI International Dead-End Metabolite Analysis of EcoCyc (2R,4S)-2-methyl-2,3,3,4- tetrahydroxytetrahydrofuran 3-hydroxypropionate 4-methyl-5-(beta- hydroxyethyl)thiazole 5,6-dimethylbenzimidazole aminoacetaldehyde cis-vaccenate cobinamide ethanolamine methanol oxamate S-adenosyl-4-methylthio-2- oxobutanoate S-methyl-5-thio-D-ribose S2- tetrahydromonapterin urate urea 148 dead-end metabolites total 16 dead-end metabolites in EcoCyc pathways:

139 © 2014 SRI International Model Debugging via Multiple Gap Filling Most FBA models are not initially solvable because of incomplete or incorrect information MetaFlux uses meta-optimization to postulate alterations to a model to render it solvable Each alteration has an associated cost; minimize cost of alterations Formulate as MILP and submit to SCIP

140 © 2014 SRI International Multiple Gap Filling of FBA Models Reaction gap filling (Kumar et al, BMC Bioinf 2007 8:212) : – Reverse directionality of selected reactions – Add a minimal number of reactions from MetaCyc to the model to enable a solution – Reaction cost is a function of reaction taxonomic range Metabolite gap filling: Postulate additional nutrients and secretions Partial solutions: Identify maximal subset of biomass components for which model can yield positive production rates

141 © 2014 SRI International MILP Objective Function for Gap Filling Σ w b B i + Σ w r R a + Σ w t R b + Σ w m R c + Σ w s S k + Σ w n N p Where W b > 0, w r, wt, w m, w s, w n < 0 are weights for biomass, reactions (2), secretions, and nutrients B i, R a, R b, R c, S k, N p are binary variables iab ckp

142 © 2014 SRI International Results – FBA Model of Human Metabolism 46 biomass compounds 13 nutrients 2secretions 207reactions carry non-zero flux

143 © 2014 SRI International MetaFlux Gap Filler Suggestions Addition of 8 new reactions from MetaCyc; 4 supported by literature research Reversal of 4 reactions confirmed by literature searches Enzyme curated into wrong compartment FBA analysis identified an amino-acid biosynthetic pathway that should not have been present in HumanCyc Further issues identified by dead-end metabolite analysis and reachability analysis

144 © 2014 SRI International

145

146 Infer Anti-Microbial Drug Targets Infer drug targets as genes coding for enzymes that encode chokepoint reactions Two types of chokepoint reactions: Chokepoint analysis of Plasmodium falciparum: – 216/303 reactions are chokepoints (73%) – All 3 clinically proven anti-malarial drugs target chokepoints – 21/24 biologically validated drug targets are chokepoints – 11.2% of chokepoints are drug targets – 3.4% of non-chokepoints are drug targets – => Chokepoints are significantly enriched for drug targets Yeh et al, Genome Research 14:917 2004

147 © 2014 SRI International Comparative Analysis Via Cellular Overview Comparative genome browser Comparative pathway table Comparative analysis reports – Compare reaction complements – Compare pathway complements – Compare transporter complements

148 © 2014 SRI International Advanced Query Form Intuitive construction of complex database queries of SQL power

149 © 2014 SRI International Other Capabilities Display and editing of protein features Blast sequences against PGDBs Retrieve nucleotide and amino acid sequences Define Web links from PGDB objects to other web sites Active community of contributors – JavaCyc, PerlCyc – SBML and BioPAX export tools

150 © 2014 SRI International Pathway Tools Implementation Details Platforms: – Macintosh, PC/Linux, and PC/Windows platforms Same binary can run as desktop app or Web server PGDBs can be stored in files, MySQL, Oracle Production-quality software – Two regular releases per year – Extensive quality assurance – Extensive documentation – Auto-patch – Automatic DB-upgrade

151 © 2014 SRI International Accesing PGDB Data Export to Genbank, SBML, BioPAX Export to tab-delimited files Export to attribute-value files Attribute-value files can be imported into SRI’s BioWarehouse – Relational database system for bioinformatics database integration APIs – Web services -- http://biocyc.org/web-services.shtml – Lisp – PerlCyc – JavaCyc

152 © 2014 SRI International Summary Pathway/Genome Databases – MetaCyc non-redundant DB of literature-derived pathways – MetaCyc family of ~4,000 PGDBs Pathway Tools software – Extract pathways from genomes – Distributed curation tools for PGDB development – Query, visualization, WWW publishing – Omics data analysis – Quantitative metabolic models

153 © 2014 SRI International Summary – Exploit Genome Information for Metabolic Engineering Create PGDB for metabolic engineering host Capture genome, metabolic network, regulatory network Extensive query and visualization tools Extensive analysis tools: – Omics and reaction flux visualization – Metabolic tracing – Comparative analysis Generate flux-balance models – Faster model generation and debugging

154 © 2014 SRI International BioCyc and Pathway Tools Availability BioCyc.org Web site and database files freely available to all Pathway Tools freely available to non-profits – Macintosh, PC/Windows, PC/Linux

155 © 2014 SRI International Acknowledgements SRI – Suzanne Paley, Ron Caspi, Mario Latendresse, Ingrid Keseler, Carol Fulcher, Tim Holland, Markus Krummenacker, Tomer Altman, Richard Billington, Pallavi Kaipa, Deepika Brito EcoCyc Collaborators – Julio Collado-Vides, Robert Gunsalus, Ian Paulsen MetaCyc Collaborators – Sue Rhee, Peifen Zhang, Kate Dreher – Lukas Mueller, Hartmut Foerster Funding sources: – NIH National Institute of General Medical Sciences – Department of Energy http://www.ai.sri.com/pkarp/talks/ BioCyc webinars: biocyc.org/webinar.shtml

156 © 2014 SRI International Learn More Pathway Tools Tutorial – April 25-27 http://bioinformatics.ai.sri.com/ptools/tutorial/


Download ppt "© 2014 SRI International Metabolomics Applications of the BioCyc Databases and Pathway Tools Software Peter D. Karpecocyc.org SRI Internationalbiocyc.org."

Similar presentations


Ads by Google