Novel Empirical FDR Estimation in PepArML David Retz and Nathan Edwards Georgetown University Medical Center

What is PepArML? Meta-search using seven search engines: Mascot; X!Tandem Native, K-Score, S-Score; OMSSA; Myrimatch; InsPecT + MSGF Automatic target + decoy searches Automatic construction of search configuration Automatic spectra and sequence (re-)formatting Heterogeneous cluster, grid, cloud computing Centralized scheduler Shared and private computational resources Integration with NSF TeraGrid and AWS 2

What is PepArML? A peptide identification result combiner Selects best identification, per spectrum Model-free, auto-train machine-learning Estimates false-discovery-rates Format output as pepXML and protXML In use: more than 23M spectra, 1.4M search jobs, and 1TB in spectra and results. PepArML identifies significantly more spectra than single search engines. Recovers more proteins with fewer replicates 3

PepArML Performance 4 LCQQSTAR LTQ-FT Standard Protein Mix Database 18 Standard Proteins – Mix1

PepArML Advantages Can accommodate new search engines or spectrum and peptide features easily Learns the specific characteristics of each dataset from scratch! Provides a platform for comparison of single search engine results with common FDR estimation procedure. 5

Search Engine Info. Gain 6

Precursor & Digest Info. Gain 7

Retention Time & Proteotypic Peptide Properties Info. Gain 8

Search Engine Independent FDR Estimation Comparing search engines is difficult due to different FDR estimation techniques Implicit assumption: Spectra scores can be thresholded Competitive vs Global Competitive controls some spectral variation Reversed vs Shuffled Decoy Sequence Reversed models target redundancy accurately Charge-state partition or Unified Mitigates effect of peptide length dependent scores What about peptide property partitions? 9

PepArML Disadvantages Training heuristic can fail to “get started” Works best on large datasets Iterative training can be time-consuming Machine-learning “confidence” is uninterpretable for peptide identification Require two decoy-searches to “calibrate” confidence as FDR Each spectrum searched ~ 21 times! 10

PepArML Disadvantages Training heuristic can fail to “get started” Works best on large datasets Iterative training can be time-consuming Machine-learning “confidence” is uninterpretable for peptide identification Require two decoy-searches to “calibrate” confidence as FDR Can we eliminate the internal decoy? Reduce search phase by 33% 11

PepArML Workflow Select high-quality IDs Guess true proteins from search results Label spectra & train Calibrate confidence Guess true proteins from ML results Iterate! Estimate FDR using (external) decoy 12

Select High-Quality Unanimous Peptide Identifications Require fast and easy, but comparable search-engine metric. 13 min decoy hitsmin z-score

Simulate Decoy Results by Sampling Target Results 14 Target Decoy Sampled Target

Simulate Decoy Results by Sampling Target Results 15 Target Decoy Sampled Target

Sampled Target Approximates Decoy Calibration Sample 75% non-training “false” target results Rescale to # of spectra Approximates FDR well- enough to replace internal decoy 16

Decoy-free PepArML results 17 LCQQSTAR LTQ-FT Standard Protein Mix Database 18 Standard Proteins – Mix1

Conclusions PepArML can significantly boost the number of spectra, peptides, and proteins identified Give it a try – free! Nothing to install! A common FDR framework facilitates head-to- head comparison of search engines and FDR estimation techniques Sampled target results can substitute for decoy results (internally) Reduces search time by 33% 18

19 Acknowledgements Growing list of PepArML users Fenselau lab (Maryland) Graham lab (JHU) Genovese lab ( Bologna University, Italy) Dr. Brian Balgley Bioproximity Dr. Chau-Wen Tseng & Dr. Xue Wu University of Maryland Computer Science Funding: NIH/NCI