1. Algorithms and Computation: Bottom-Up Data Analysis Workflows
Nathan Edwards
Georgetown University Medical Center

2. Changing landscape Experimental landscape Computational landscape
Spectra, sensitivity, resolution, samples
Computational landscape
Data-size, cloud, resources, reliability
Data-size and false positive identifications
Controlling for false proteins/genes
Improving peptide identification sensitivity
Machine-learning, multiple search engines
Filtered PSMs as a primary data-type

3. Changing Experimental Landscape
Instruments are faster…
More spectra, better precursor sampling
Sensitivity improvements…
More fractionation (automation), deeper precursor sampling, ion optics
Resolution continues to get better…
Accurate precursors (fragments) make a big difference
Analytical samples per study…
Fractionation, chromatography, automation improvements

4. Clinical Proteomic Tumor Analysis Consortium (NCI)
Comprehensive study of genomically characterized (TCGA) cancer biospecimens by mass-spectrometry-based proteomics workflows
~ 100 clinical tumor samples per study
Colorectal, breast, ovarian cancer
CPTAC Data Portal provides
Raw & mzML spectra; TSV and mzIdentML PSMs; protein reports; experimental meta-data

5. …from Edwards et al., Journal of Proteome Research, 2015
CPTAC Data Portal
…from Edwards et al., Journal of Proteome Research, 2015

6. CPTAC/TCGA Colorectal Cancer (Proteome)
Vanderbilt PCC (Liebler)
95 TCGA samples, 15 fractions / sample
Label-free spectral count / precursor XIC quant.
Orbitrap Velos; high-accuracy precursor
1425 spectra files ~600 Gb/~129 Gb (mzML.gz)
Spectra: ~ 18M; ~ 13M MS/MS
~ 4.6M PSMs at 1% MSGF+ q-value

7. Changing Computational Landscape
Single computer operating on a single spectral data-file is no longer feasible
MS/MS search is the computational bottleneck
Private computing clusters are quickly obsolete
Need $$ to upgrade every 3-4 years
Personnel costs for cluster administration and management
Cloud computing gets faster and cheaper over time…
…but requires rethinking the computing model

8. PepArML Meta-Search Engine
Simple, unified, peptide identification search parameterization and execution:
Mascot, MSGF+, X!Tandem, K-Score, S-Score, OMSSA, MyriMatch
Cluster, grid, and cloud scheduler:
Reliable batch spectra conversion and upload,
Automated distribution of spectra and sequence,
Job-failure tolerant with result-file validation
Machine-learning-based result combining:
Model-free – heterogeneous features
Adapts to the characteristics of each dataset

9. PepArML Meta-Search Engine
Georgetown
& Maryland HPC
Heterogeneous
compute resources
Secure
communication
Edwards Lab
Scheduler &
48+ CPUs
Under the hood, the user interacts with a scheduler, uploading spectra, and specifying a meta-search. The compute clients, local or remote, contact the scheduler to get spectra and jobs to compute.
Single, simple search request
Amazon Web Services

10. PepArML Meta-Search Engine
Georgetown
& Maryland HPC
Heterogeneous
compute resources
Secure
communication
Edwards Lab
Scheduler &
48+ CPUs
Under the hood, the user interacts with a scheduler, uploading spectra, and specifying a meta-search. The compute clients, local or remote, contact the scheduler to get spectra and jobs to compute.
Single, simple search request
Amazon Web Services

11. Run all of the search engines!

12. Search Engine Running Time
Which (combination of) search engine(s) should I use?

13. Fault Tolerant Computing
Spot instances can be preempted for those willing to pay more
Spot prices are cheaper (7¢/hour vs 46¢/hour)

14. Identifications per $$
How long will a specific job take?
How much memory / data-transfer is needed?
What is a good decomposition size?
What cloud-instance to use?
Wall-clock time can be significantly reduced:
…but management overhead costs too.
Cost of total compute may even increase.
Failed analyses cost too!

15. Data-Scale and False Positives
Big datasets have more false positive proteins and genes!
CPTAC Colorectal Cancer (CDAP)
4.6M MSGF+ 1% FDR PSMs + 2 peptides/gene
~ 10,000 genes identified…

16. Data-Scale and False Positives
Big datasets have more false positive proteins and genes!
CPTAC Colorectal Cancer (CDAP)
4.6M MSGF+ 1% FDR PSMs + 2 peptides/gene
~ 10,000 genes identified… …but ~ 40% gene FDR

17. Simple decoy protein model
Decoy peptides hit decoy proteins uniformly.
Each decoy peptide represents an independent trial.
Binomial distribution on
size of protein database
number of decoy peptides
Big-datasets have more decoy peptides!

18. Example Large: 10,000 proteins, 100,000 peptides
Small: 1,000 proteins, 10,000 peptides

19. Data-Size and False Positives
CPTAC Colorectal Cancer
1% FDR PSMs, but ~ 25% peptide FDR
~ 25,000 decoy peptides on ~ 20,000 genes
Control of gene FDR requires even more stringent filtering of PSMs.
If we require strong evidence in all 95 samples:
No decoy genes, but less than 1000 genes identified.
Bad scenario:
PDHA1 and PDHA2 in CPTAC Breast Cancer – shared and unique peptides
PDHA2 is testes specific!

20. Improved Sensitivity Machine-learning models
Use additional metrics for good identifications
Combining multiple search engines
Agreement indicates good identifications
Both approaches successful at boosting ids, particularly when adaptable to each dataset.
Watch for the use of decoys in training the model.
Both have scaling issues and lack transparency
…may add noise to comparisons

21. PepArML Performance Standard Protein Mix Database
LCQ
QSTAR
LTQ-FT
Standard Protein Mix Database
18 Standard Proteins – Mix1

22. Search Engine Info. Gain

23. Precursor & Digest Info. Gain

24. Filtered PSMs as Primary Data
For large enough spectral datasets, we might choose best effort peptide identification
Filtered PSMs become primary data
Spectral counts become more quantitative
Need linear-time spectra → PSM algorithm
We work less hard to identify all spectra?
Output as genome alignments, BAM files?
How should PSMs be represented to maximize their utility?
What about decoy peptide identifications?

25. Nascent polypeptide-associated complex subunit alpha

26. Pyruvate kinase isozymes M1/M2
2.5 x 10-5

27. Questions?