Presentation is loading. Please wait.

Presentation is loading. Please wait.

AB RF Proteome Informatics Research Group iPRG 2012: A Study on Detecting Modified Peptides in a Complex Mixture ABRF 2012, Orlando, FL 3/17-20/2012.

Similar presentations


Presentation on theme: "AB RF Proteome Informatics Research Group iPRG 2012: A Study on Detecting Modified Peptides in a Complex Mixture ABRF 2012, Orlando, FL 3/17-20/2012."— Presentation transcript:

1 AB RF Proteome Informatics Research Group iPRG 2012: A Study on Detecting Modified Peptides in a Complex Mixture ABRF 2012, Orlando, FL 3/17-20/2012

2 AB RF Proteome Informatics Research Group IPRG2012 STUDY: DESIGN 2

3 AB RF Proteome Informatics Research Group Study Goals Primary:Evaluate the ability of participants to identify modified peptides present in a complex mixture Secondary:Find out why result sets might differ between participants Tertiary:Produce a benchmark dataset, along with an analysis resource 3

4 AB RF Proteome Informatics Research Group Study Design Use a common, rich dataset Use a common sequence database Allow participants to use the bioinformatic tools and methods of their choosing Use a common reporting template Report results at an estimated 1% FDR (at the spectrum level) Ignore protein inference 4

5 AB RF Proteome Informatics Research Group Sample Tryptic digest of yeast (RM8323 – NIST), spiked with 69 synthetic modified peptides (tryptic peptides from 6 different proteins – sPRG) – Phospho (STY) – Sulfo (Y) – Mono-, di-, trimethyl (K) – Mono-, dimethyl (R) – Acetyl (K) – Nitro (Y) 5

6 AB RF Proteome Informatics Research Group Supplied Study Materials 5600 TripleTOF dataset (i.e. WIFF file) – WIFF, mzML, dta, MGF (de-isotoped);– conversions by MS Data Converter – MGF (not de-isotoped – conversion by Mascot Distiller 2.4) 1 fasta file (UniProtKB/SwissProt S. cerevisiae, human, + 1 bovine protein + trypsin from Dec. 2011) 1 template (Excel) 1 on-line survey (Survey Monkey) 6

7 AB RF Proteome Informatics Research Group Instructions to Participants 1.Retrieve and analyze the data file in the format of your choosing, with the method(s) of your choosing 2.Report the peptide to spectrum matches in the provided template 3.Report measures of reliability for PTM site assignments (optional) 4.Fill out the survey 5.Attach a 1-2 page description of the methodology employed 7

8 AB RF Proteome Informatics Research Group iPRG 2012 STUDY: PARTICIPATION 8

9 AB RF Proteome Informatics Research Group Study advertised on the ABRF website and listserv and by direct invitation from iPRG members 1. participation request to 2. Send official study letter with instructions 3. All further communication (e.g., questions, submission) through iPRG members Participant Questions / Answers Anonymizer Soliciting Participants and Logistics 9

10 AB RF Proteome Informatics Research Group Participants (i) – overall numbers 24 submissions – One participant submitted two result sets 9 initialed iPRG member submissions (with appended i) 2 vendor submissions (identifiable by appended v) 10

11 AB RF Proteome Informatics Research Group About the Participant 11

12 AB RF Proteome Informatics Research Group About the Participants Lab 12

13 AB RF Proteome Informatics Research Group Participation in sPRG Study 13 Only one participant indicated he used sPRG information to aid his analysis. This person was one of the least successful in identifying the spiked-in peptides!

14 AB RF Proteome Informatics Research Group Search Engine Used 14

15 AB RF Proteome Informatics Research Group Site Localization Software 15 4 participants did not list using software for site localization.

16 AB RF Proteome Informatics Research Group Summary of Submitted Results 16 Only reported modified peptides

17 AB RF Proteome Informatics Research Group Summary of IDs and Localizations 17 Peptide Identification in all Spectra Site Localization in Spectra With Interesting Modifications

18 AB RF Proteome Informatics Research Group Overlap of spectrum identifications agreed on by 3 or more participants

19 AB RF Proteome Informatics Research Group AnAndromeda/MaxQuantMGMS-GFDBpFpFindScScaffold ASA-ScoreMMMyriMatchPkPEAKSSMSpectrum Mill ByByonicsMOMODaPkDBPEAKSDBSqSequest IHIn-house softwareOOMSSAPPiProtein PilotSTSpectraST IPIDPickerOtOtherPPrProtein ProspectorTPPTransProteomic Pipeline MMascotP/PPPep/Prot ProphetPRPhosphoRSXLExcel MDeMascot Delta ScorePDProteomeDiscovererPWProteoWizardXTX!Tandem MDiMascot Distiller Room for improvement in thresholding? v58288v i i94158i97053i42424i77777i i87048i i PeaklistmgfmzML mgf mzMLmgf_ndmgfmgf_ndmgfmzML mgf_ndmgf mzMLmgfmgf_ndmgf mgf_ndmgfmgf_nd mzML mgf_nd WIFF Spectral Pre-ProcessingPk PPiMDiOtSMpFPkMDiPD PkDBPWPkDBSq Peptide IdentificationByPk MPPiMOPPrpFMMGMP/PPSMpFMPkMPPiMOMMPD PkDB OSTMMOTPPPkDBSq STXTIHXT Ot Discovery of Unexpected ModsByPk MPPiSTPPrpFIHMOSMpFMPkPPiOM PkDB Site LocalizationPk MDePPiMPPrpFMIHMSMpFMPkMASIHAnMDePD PkDBOMMOPkDBScOtSq STIH Results FilteringByPk IHPPiP/PP PPrpFIPIHXLIHXLpFMPkMXLScMPR PkDBXLOtTPP XLPkDB NTT21121?11121?12222?1222?2 Experience 5-10 years >10 years< 1 year3-4 years 5-10 years >10 years 5-10 years 3-4 years 5-10 years >10 years1-2 years >10 years1-2 years >10 years 1-2 years >10 years 1-2 years >10 years

20 AB RF Proteome Informatics Research Group ESR and FDR Extraordinary Skill Rate or High False Discovery Rate? ESR + FDR = 100* (Y<3P+YD)/total ids Y participants 3 for consensus

21 AB RF Proteome Informatics Research Group Characteristics of consensus spectra 7840 spectra >=3 participants agreeing on sequence Consensus requires agreement on Sequence, but not modification localization 21

22 AB RF Proteome Informatics Research Group Peak lists Two types of peak lists were supplied – Deisotoped and non deisotoped Can only tell fragment charge state from non- deisotoped Requires search engine to be able to de-isotope spectrum 22

23 AB RF Proteome Informatics Research Group Peaklists 23 Number of spectra with undefined precursor charge state Deisotoped 1031 (304 in consensus results) Non-deisotoped 6094 (1140 in consensus results) For 1013 out of 7840 consensus spectra the precursor m/z differ by greater than 0.02 Da between deisotoped and non-deisotoped peak list. For 238 consensus spectra the peak lists had different specified charge state – 193 consensus results only possible with deisotoped peak list – 45 consensus results only possible with non-deisotoped peak list – For 19 consensus results multiple people who searched the nd peak list agreed on a confident different answer – For 4 consensus results multiple people who searched the deisotoped peak list agreed on a confident different answer

24 AB RF Proteome Informatics Research Group Mixed Spectra Non-deisotoped peaklist Deisotoped peaklist

25 AB RF Proteome Informatics Research Group Synthetic Peptide ID by Peptide 25 Sulfo Phospho Nitro Trimethyl Methyl (R) Dimethyl (R) Acetyl Methyl (K) Dimethyl (K) # participants

26 AB RF Proteome Informatics Research Group Synthetic Peptide ID by Participant v58288v i i94158i97053i42424i77777i i87048i i Acetyl (K) Dimethyl (K) Dimethyl (R) Methyl (K) Methyl (R) Nitro (Y) Phospho (STY) Sulfo (Y) Trimethyl (K)

27 AB RF Proteome Informatics Research Group Correct Localization of Modified Synthetic Peptides synthetic modified peptides were spiked into sample. 7 of these were confidently found by no participant Correct localization & name of modification reported

28 AB RF Proteome Informatics Research Group FLR of Modified Synthetic Peptides 28 FLR = 100% * # PSMs wrong localization of s,t,y,k,r # PSMs wrong + right localization of s,t,y,k,r Ignored PSMs contain mods of residues other than s,t,y,k,r. Sample handling mods (n,q,d,e, etc). 5% 1%1-2% 1%<10.510% <30% 0.01<5% <1%

29 AB RF Proteome Informatics Research Group Incorrect Localization by Peptide v58288v i i94158i97053i42424i77777i i87048i i EKLLDFIK AEGSEIRLAK1 VDATEESDLAQQYGVR TITLEVEPSDTIENVK ESTLHLVLR AEFAEVSK1 LKLVSELWDAGIK DQGGELLSLR TYETTLEK1 NGDTASPKEYTAGR LKAEGSEIR TVIDYNGER ADEGISFR YKPESDELTAEK GTRDYSPR1 VPQVSTPTLVEVSR ADEGISFRGLFIIDDK ALAPEYAK TIAQDYGVLK THILLFLPKSVSDYEGK WVTFISLLFLFSSAYSR IFSIVEQR TLSDYNIQK31 GILRQITVNDLPVGR NVAVDELSR LDELRDEGK ESTLHLVLRLR1 DEGKASSAK SVSDYEGK LVQAFQFTDK LVNEVTEFAK1 GLFIIDDKGILR1 FPKAEFAEVSK LKAQLGPDESK DISLSDYK FKDLGEENFK Number of PSMs with Incorrect Site Localization – Mod Loc Confidence Y Present as sulfo-Tyr Present as phospho S-10 often mislocalized as S-12 or Y-14 Present as mono, di, tri methyl K often mislocalized at R

30 AB RF Proteome Informatics Research Group 30 DISLSDY(Phospho)K DISLSDY(Sulfo)K Observe modified fragment ions. Observe unmodified fragment ions. Spectrum looks essentially identical to unmodified peptide spectrum Phospho vs Sulfo

31 AB RF Proteome Informatics Research Group Conclusions Reasonable number of participants from around the globe, mainly experienced users but a few first-timers Large spread in number of spectra identified False negatives (NS) are generally much higher than false positives, so there is generally room for improvement Peak list was a significant factor on performance Varied performance in detecting PTMs Most participants struggled with sulfation Multiply phosphorylated harder to find than singly Most common errors in site assignment were: Reporting sulfo(Y) as phospho(ST) Mis-assignment of site/s in multiply phosphorylated peptides 31

32 AB RF Proteome Informatics Research Group What did the participants think? 22 out of 24 participants found the study useful 32 Too many modifications at the same time. Manual validation is necessary and the right time necessary for this study is too demanding for this challenge. The spiked proteins made it possible to game the study - look for the uncommon modifications only on the spikes. Of course we didn't do this. Overall I'd say this was a flawed but very interesting ABRF study.

33 AB RF Proteome Informatics Research Group 33 Before After Participants Confidence in Analyzing PTM Data

34 AB RF Proteome Informatics Research Group 34 How difficult do you think this study was? What was your total analysis time for the entire project?

35 AB RF Proteome Informatics Research Group 35 Based on this study, would you consider participating in future ABRF studies?

36 AB RF Proteome Informatics Research Group Thank you! Questions? iPRG Nuno Bandeira Robert Chalkley(chair) Matt Chambers Karl Clauser John Cottrell Eric Deutsch Eugene Kapp Henry Lam Hayes McDonald Tom Neubert (EB liaison) Ruixiang Sun Dataset Creation Chris Colangelo Anonymizer: Jeremy Carver, UCSD THANK YOU TO ALL STUDY PARTICIPANTS! 36


Download ppt "AB RF Proteome Informatics Research Group iPRG 2012: A Study on Detecting Modified Peptides in a Complex Mixture ABRF 2012, Orlando, FL 3/17-20/2012."

Similar presentations


Ads by Google