Presentation is loading. Please wait.

Presentation is loading. Please wait.

Scaling Species Tree Estimation to Large Datasets

Published byVictoria Godoy Modified over 4 years ago

Similar presentations

Presentation on theme: "Scaling Species Tree Estimation to Large Datasets"— Presentation transcript:

1 Scaling Species Tree Estimation to Large Datasets
Tandy Warnow The University of Illinois

2 Phylogeny + genomics = genome-scale phylogeny estimation
.

3 Phyogenomic Pipeline Assemble and annotate genomes (e.g., determine orthologs) Compute multiple sequence alignments of individual loci Construct gene trees Construct species tree Perform post-tree analyses (e.g., estimate dates, infer selection, etc.)

4 Phyogenomic Pipeline Assemble and annotate genomes (e.g., determine orthologs) Compute multiple sequence alignments of individual loci Construct gene trees Construct species tree Perform post-tree analyses (e.g., estimate dates, infer selection, etc.)

5 Phyogenomic Pipeline Assemble and annotate genomes (e.g., determine orthologs) Compute multiple sequence alignments of individual loci Construct gene trees Construct species tree Perform post-tree analyses (e.g., estimate dates, infer selection, etc.)

6 PASTA and UPP scaling MSA to 1,000,000 sequences
PASTA (Mirarab et al., J. Computational Biology 2014) – an improvement over SATe (Liu et al., Science 2009) PASTA available at PASTA+BAli-Phy at UPP (Nguyen et al., Genome Biology 2015) – uses PASTA for backbone, inserts sequences into the backbone using ensembles of HMMs, handles fragmentary sequences well

7 Phyogenomic Pipeline Assemble and annotate genomes (e.g., determine orthologs) Compute multiple sequence alignments of individual loci Construct gene trees Construct species tree Perform post-tree analyses (e.g., estimate dates, infer selection, etc.)

8 Phyogenomic Pipeline Assemble and annotate genomes (e.g., determine orthologs) Compute multiple sequence alignments of individual loci Construct gene trees Construct species tree Perform post-tree analyses (e.g., estimate dates, infer selection, etc.)

9

10 Multiple causes for discord, including
Incomplete Lineage Sorting (ILS), Gene Duplication and Loss (GDL), and Horizontal Gene Transfer (HGT)

11 Multiple causes for discord, including
Incomplete Lineage Sorting (ILS), Gene Duplication and Loss (GDL), and Horizontal Gene Transfer (HGT)

12 Main competing approaches
gene gene gene k . . . Species Concatenation . . . Analyze separately point out that supertree methods take overlaping trees and produce a tree, and that the whole process of first generating small trees and then applying a supertree method is often referred to as the “supertree approach”. Summary Method

13 ASTRAL Mirarab and Warnow, Bioinformatics 2014
Algorithmic approach: Given set of gene trees, find the species tree that agrees with the maximum number of quartet trees within a constrained search space. Polynomial time and statistically consistent in the presence of ILS.

14 ASTRID ASTRID: Accurate species trees using internode distances, Vachaspati and Warnow, RECOMB-CG 2015 and BMC Genomics 2015 Github site: Algorithmic design: Nearly the same as NJst (Liu and Yu, 2010)- computes a matrix of average leaf-to-leaf topological distances, and then computes a tree using FastME (more accurate than Neighbor Joining and faster, too). Polynomial time and statistically consistent in the presence of ILS.

15

16

17 Impact of Gene Tree Estimation Error (from Molloy and Warnow 2017)
Note: Summary methods better than CA-ML for low GTEE, then worse! Error is fraction of bipartitions that are not recovered Summary Methods Site-based Method

18 Main competing approaches
gene gene gene k . . . Species Concatenation . . . Analyze separately point out that supertree methods take overlaping trees and produce a tree, and that the whole process of first generating small trees and then applying a supertree method is often referred to as the “supertree approach”. Summary Method

19 Large datasets are challenging!
RAxML is expensive! 250 CPU years for the Avian Phylogenomics project with only 48 species Not guaranteed to find global optimum, in any event And concatenation analyses are not statistically consistent ASTRAL can also fail to complete in reasonable times for very challenging datasets (very high ILS, many taxa), though not as problematic as RAxML

20 Our Approach Divide-and-conquer pipeline to: Improve scalability,
reduce running time, and maintain accuracy for species tree estimation on large datasets

21 Divide-and-Conquer Pipeline
Decompose species set into pairwise disjoint subsets. Full species set Build a tree on each subset Compute tree on entire set of species using “Disjoint Tree Merger” method Tree on full species set Auxiliary Info (e.g., distance matrix)

22 TreeMerge Molloy and Warnow, ISMB 2019 and Bioinformatics, Volume 35, Issue 14, July 2019, Pages i417–i426 TreeMerge is a direct improvement to NJMerge (which it uses on pairs of trees) Github site: Algorithmic strategy: divide species set into disjoint subsets, compute species trees on the subsets using selected species tree method, and merge subset trees using a distance-based method

23 DTMs Merge Subset Trees
Fig. 1. Given a set of trees on pairwise disjoint leaf sets, many possible compatibility supertrees exist. Here we show two compatibility supertrees, labeled Ti,j,k and Ti,j,k′, for T={Ti,Tj,Tk}; others can also be identified. Notably, the trees in T form connected subtrees in Ti,j,k (i.e. Ti,j,k is formed by connecting the trees in T by edges), but this is not the case for Ti,j,k′. We refer to the first type of compatibility supertree as un-blended, and the second type as blended Unless provided in the caption above, the following copyright applies to the content of this slide: © The Author(s) Published by Oxford University Press.This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License ( which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact Notes: Subset trees are requirements (constraint trees) Blending is permitted! Bioinformatics, Volume 35, Issue 14, July 2019, Pages i417–i426, The content of this slide may be subject to copyright: please see the slide notes for details.

24 TreeMerge study Datasets: 1000 species and 1000 genes (exons and introns), two levels of ILS (low/mod and very high) Base methods: ASTRAL and with RAxML Criteria: Impact on base method How many replicates complete? Species tree error Running time?

25 Impact of using TreeMerge with RAxML on 1000 species and 1000 genes
Fig. 4. Impact of using TreeMerge-fast with RAxML. The top row shows species tree estimation error for datasets with 1000 taxa and 1000 genes; note that gray bars represent medians, gray squares represent means, gray circles represent outliers, box plots extend from the first to the third quartiles, and whiskers extend to plus/minus 1.5 times the interquartile distance (unless greater/less than the maximum/minimum value). The bottom row shows running time (in minutes); bars represent means and error bars represent standard deviations across replicate datasets. The running time of TreeMerge-fast is the time to estimate the distance matrix, to estimate each subset tree using RAxML, and to combine the subset trees using TreeMerge-fast (Equation 1). The number N of replicates on which RAxML completed is shown on the x-axis; note that averages are taken across the replicates on which RAxML completed. When RAxML did not complete, it was due to OOM errors; otherwise the last checkpoint written by RAxML was evaluated Unless provided in the caption above, the following copyright applies to the content of this slide: © The Author(s) Published by Oxford University Press.This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License ( which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact Bioinformatics, Volume 35, Issue 14, July 2019, Pages i417–i426, The content of this slide may be subject to copyright: please see the slide notes for details.

26 Impact of using TreeMerge with ASTRAL-III on 1000 species and 1000 genes
Fig. 3. Impact of using TreeMerge-fast with ASTRAL-III. The top row shows species tree estimation error for datasets with 1000 taxa and 1000 genes; gray bars represent medians, gray squares represent means, gray circles represent outliers, box plots extend from the first to the third quartiles, and whiskers extend to plus/minus 1.5 times the interquartile distance (unless greater/less than the maximum/minimum value). The bottom row shows running time (in minutes); bars represent means and error bars represent standard deviations across replicate datasets. The running time of TreeMerge-fast is the time to estimate the distance matrix, to estimate each subset tree using ASTRAL-III, and to combine the subset trees using TreeMerge-fast (Equation 1). The number N of replicates on which ASTRAL-III completed is shown on the x-axis; note that averages are taken across the replicates on which ASTRAL-III completed. When ASTRAL-III did not complete, it was due to running longer than the 48-h maximum wall-clock time Unless provided in the caption above, the following copyright applies to the content of this slide: © The Author(s) Published by Oxford University Press.This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License ( which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact Bioinformatics, Volume 35, Issue 14, July 2019, Pages i417–i426, The content of this slide may be subject to copyright: please see the slide notes for details.

27 Summary for TreeMerge TreeMerge
Improves scalability (ASTRAL and RAxML can fail to complete in 48 hours, but not when run with TreeMerge) Reduces running time, sometimes dramatically Maintains accuracy

28 Other DTMs NJMerge: Molloy and Warnow, RECOMB-CG 2018 and Alg Mol Biol (to appear) Constrained-INC: Zhang, Rao, and Warnow, WABI 2018, Le et al., AlCoB 2019, and submitted GTM (Guide Tree Merger): Smirnov and Warnow, RECOMB-CG 2019 and BMC Genomics (submitted) Similar accuracy to TreeMerge in most cases, but faster Does not permit blending

29 Summary Recommendations for large and ultra-large datasets:
MSA: PASTA and UPP Gene trees: ML methods, such as RAxML Species trees: ASTRAL, ASTRID, or concatenation using ML Ultra-large datasets? Use a Disjoint Tree Merger method, such as TreeMerge Downloadable software: GTM: TreeMerge: PASTA: UPP: ASTRAL: ASTRID:

30 Acknowledgments Software: all open source, see NSF grants DBI , ABI , CCF NSF graduate fellowships to Pranjal Vachaspati and Erin Molloy HHMI graduate fellowship to Siavash Mirarab Computing done at Blue Waters (part of NCSA) Papers: Presentations:

31 Accuracy in the presence of HGT + ILS
Davidson et al., RECOMB-CG, BMC Genomics 2015

Download ppt "Scaling Species Tree Estimation to Large Datasets"

Similar presentations

New methods for estimating species trees from genome-scale data Tandy Warnow The University of Illinois.

New methods for estimating species trees from genome-scale data Tandy Warnow The University of Illinois.
New methods for estimating species trees from genome-scale data Tandy Warnow The University of Illinois.

New methods for estimating species trees from genome-scale data Tandy Warnow The University of Illinois.
Using Divide-and-Conquer to Construct the Tree of Life Tandy Warnow University of Illinois at Urbana-Champaign.

Using Divide-and-Conquer to Construct the Tree of Life Tandy Warnow University of Illinois at Urbana-Champaign.
CS 598 AGB Supertrees Tandy Warnow. Today’s Material Supertree construction: given set of trees on subsets of S (the full set of taxa), construct tree.

CS 598 AGB Supertrees Tandy Warnow. Today’s Material Supertree construction: given set of trees on subsets of S (the full set of taxa), construct tree.
Scaling BAli-Phy to Large Datasets June 16, 2016 Michael Nute 1.

Scaling BAli-Phy to Large Datasets June 16, 2016 Michael Nute 1.
Fig. 1. Genomic structure of the csd gene in A

Fig. 1. Genomic structure of the csd gene in A
Fig. 8 The two cases for the terminal edge e<sub>0</sub>

Fig. 8 The two cases for the terminal edge e<sub>0</sub>
Fig. 1. — The life cycle of S. papillosus. (A) The life cycle of S

Fig. 1. — The life cycle of S. papillosus. (A) The life cycle of S
From: More on the Best Evolutionary Rate for Phylogenetic Analysis

From: More on the Best Evolutionary Rate for Phylogenetic Analysis
Fig. 1. Trial structure for the experimental task.

Fig. 1. Trial structure for the experimental task.
Fig. 1. RUbioSeq pipelines for exome variant detection and BS-Seq analyses. Dark gray boxes correspond to the main steps of the pipelines. Light gray boxes.

Fig. 1. RUbioSeq pipelines for exome variant detection and BS-Seq analyses. Dark gray boxes correspond to the main steps of the pipelines. Light gray boxes.
Mathematical and Computational Challenges in Reconstructing Evolution

Mathematical and Computational Challenges in Reconstructing Evolution
Tandy Warnow The University of Illinois

Tandy Warnow The University of Illinois
Large-Scale Multiple Sequence Alignment

Large-Scale Multiple Sequence Alignment
Mathematical and Computational Challenges in Reconstructing Evolution

Mathematical and Computational Challenges in Reconstructing Evolution
Figure 1: Necrobiosis lipoidica: Yellow-brown skin lesions with indurated borders located on both lower legs. Figure 1: Necrobiosis lipoidica: Yellow-brown.

Figure 1: Necrobiosis lipoidica: Yellow-brown skin lesions with indurated borders located on both lower legs. Figure 1: Necrobiosis lipoidica: Yellow-brown.
Figure 1. Circular taxonomy tree based on the species that were sequenced in our study. Unless provided in the caption above, the following copyright applies.

Figure 1. Circular taxonomy tree based on the species that were sequenced in our study. Unless provided in the caption above, the following copyright applies.
Figure 1: Axial T2 W images of penis showed a well-defined septated area of 2 cm in the posterior aspect of ... Figure 1: Axial T2 W images.

Figure 1: Axial T2 W images of penis showed a well-defined septated area of 2 cm in the posterior aspect of ... Figure 1: Axial T2 W images.
FIGURE 1 Histological diagnoses divided into 8-year time frames (n = 1208). Unless provided in the caption above, the following copyright applies to the.

FIGURE 1 Histological diagnoses divided into 8-year time frames (n = 1208). Unless provided in the caption above, the following copyright applies to the.
Figure 1. Position and number of NLS improves genome editing by AsCas12a, LbCas12a and FnoCas12a. (A) General schematic ... Figure 1. Position and number.

Figure 1. Position and number of NLS improves genome editing by AsCas12a, LbCas12a and FnoCas12a. (A) General schematic ... Figure 1. Position and number.

Similar presentations

Ads by Google