Table 1. Quality Parameters Being Considered for Evaluation Developing Procedures to Benchmark and Optimize the Performance of Mass Spectrometers in Individual Proteomics Laboratories Emily Chen1, LeeAnn Higgins2, Achim Treumann3, Sheng Zhang4, Theresa McLaughlin5, Allis Chien5 1Columbia University, New York, NY; 2University of Minnesota, St Paul, MN; 3Newcastle University, Newcastle, UK; 4Cornell University, Ithaca, NY; 5Stanford University, Stanford, CA Overview The Workflow Interest Network (WIN) is an ABRF Research Group composed of volunteer scientists who believe in inter-laboratory repeatability and reproducibility. The WIN's mission is to help scientists and resource facilities improve reproducibility of scientific data by optimizing analytical parameters and providing recommendations for reproducible inter-laboratory workflows. Initially the group will focus on qualitative and quantitative experiments using mass spectrometry based technology. Ultimately, we welcome scientists from other types of technological platform to join our efforts and share their experiences of developing reproducible workflows in their fields. ABRF 2017 Annual Meeting March 25-28 2017 Introduction Preliminary Results Future In this ABRF study we are developing a set of robust procedures that simplify instrument performance optimization among proteomics core facilities independent of instrument platform and geographic location. Commercially available standardized mixtures are relatively affordable and broadly available. A workflow that simplifies the collection and analysis of system suitability data using these standard mixtures would be useful for the proteomics community. In phase 2 of our study, we will recruit >40 participating laboratories, provide a template for each type of instrument derived from the phase 1 study, and ask each participant to perform an experiment similar that from phase 1. We will collect raw files and instrument/LC parameters from each participating laboratory, perform ID-free matrix multivariate analysis, compare ID-free analysis with actual protein identification data, and identity factors that have the most impact on inter-laboratory variability. Retention Time Reproducibility Mass Accuracy Lab 4 Lab 6 Lab 6 Lab 1-ref Lab 2 Lab 1-ref Lab 4 Lab 5 Lab 5 Lab 2 Lab 3 Lab 3 Fusion/Lumos Fusion/Lumos Table 1. Quality Parameters Being Considered for Evaluation ID-Free Metrics Sample Monitoring Parameters Information Peptide RT mix std Retention Time (RT) LC stability FWHM LC performance - peptide separation TIC/Base peak LC reproducibility MS1 mass accuracy Instrument performance MS2 mass accuracy Peak intensity Median ms2 injection time Digested cell lysate std Retention Time (RT) of spiked peptide stds MS2 Repeat Sampling Frequency ID-Dependent Metrics # of identified PSM (fixed FDR & search algorithm) Charge distribution of peptides # of missed cleavages (proteolysis) Sample preparation Materials and Methods Aliquots of Pierce Peptide RT mix and Hela cell lysate digest were analyzed. Participating laboratories were asked to Clean the LC column, clean and calibrate the instrument, then perform 5 injections of the 15 peptide RT mix (10fmole on column) 1 injection of the Digested Hela lysate spiked with RT mix peptides (100ng digest + 10fmol peptide mix, 100 min suggested gradient) 1 injection of the Peptide RT mix 1 injection blank water 1 injection Hela lysate 1 injection Peptide RT mix Participants were asked to generate a dataset using a standardized HPLC gradient. However, they were also encouraged to generate a dataset using their own optimized gradient if they believed it would provide better quality data. Lab 3 Lab 4 Lab 5 Lab 2 Lab 5 Lab 1-ref Lab 1-ref Lab 2 Lab 3 Lab 4 Elite/Velos Elite/Velos Lab 4 Lab 3 Lab 4 Lab 3 Lab 1-ref Lab 5 Lab 1-ref Lab 2 Lab 5 Lab 2 QE/Other QE/Other Chromatographic retention time (RT) is a quality parameter that can be readily observed. RTs vary among instrument systems, in part due to variations in dead volume. However, RTs for runs on the same instrument are quite reproducible. Deviations indicating column contamination or other system issues can be addressed promptly. Other quality parameters such as mass accuracy, fill time, or Peptide Spectral Matches (see Table 1) are difficult to monitor without specialized software and/or data processing. Issues affecting data acquisition take longer to be recognized and addressed with currently available software tools and workflows. Participating Laboratories 11 laboratories in the Americas and Europe participated in Phase 1 of the study. Instrument platforms included the Orbitrap Elite, Fusion, Velos, Q-Exactive, and Triple TOF. Acknowledgements The Workflow Interest Group (WIN) thanks Thermo Fisher Scientific and Biognosys for providing materials used in this study. References Walzer M et al, Mol Cell Proteomics, 2014 Aug;13(8):1905-13 Kinsinger C et al, Mol Cell Proteomics, 2011 Nov; 10(12): mcp.O111.015446