Cosmic Program and Users in the Fermilab Dark Energy program Brenna Flaugher Fermilab Institutional Review Feb. 2015

Cosmic Dark Energy Program Fermilab plays a critical role in Cosmic community by providing the resources and management experience that brings together Universities and Labs to build new projects, in particular projects that are large on the traditional scale of cosmic projects In the 90’s Fermilab got involved in SDSS, critical role in pushing it to completion, management, computing, engineering – the dawn of the age of Big Surveys, and Big publically accessible survey Data DES – currently ~300 people, Fermilab led the construction of the camera, and is leading the operations DESI – collaboration ~150 and still growing, LBL is leading with strong Fermilab support, (Project scientist, CCDs, Barrel) LSST – SLAC leading, Fermilab engaged in DESC science and working towards a role in operations Fermilab Institutional Review 2015

DECam and DES – Example of Lab-University collaboration The Dark Energy Survey Collaboration formed to build DECam and to perform DES. The collaboration started forming in 2003, Fermilab led, Fermilab staff fill key roles: –John Peoples - DES Director until 2010, Now Josh Frieman –Project Manager - B.F. (project completed in 2012) –Operations Scientist - Tom Diehl –Data Management Project Scientist Brian Yanny The collaboration of universities and labs was built around the principle that each institution brought critical expertise and/or resources to the project in exchange for collaboration membership and access to the data and analysis tools. Collaboration is governed by the management committee, with representatives from each institution. The MC was engaged in decisions on institutional roles in the project Fermilab Institutional Review 2015

Technical contributions to the DECam project were distributed across the DES collaboration (also true for DESI) Fermilab –Management –Mechanical / Electrical engineering –CCD packaging and Testing (also on DESI) –Corrector barrel design, lens cell alignment (also on DESI) –Integration and assembly Key University contributions –OSU: Online software, observer interfaces (also on DESI) –UIUC: fabrication of front end electronics crates –Michigan: design and fabrication of the filter changer –TAMU: Spectrophotometric Calibration system, ATMCam, GPSmon –UCL: Optical expertise, optical corrector assembly (also on DESI) –Barcelona: Guiding and CCD readout electronics (also on DESI) –Madrid: CCD readout electronics –UChicago: Multi-CCD Test Vessel Key contributions from other labs –LBNL: CCD production (also on DESI) –ANL: Mechanical engineering (F/8, TS Rings), Slow controls –SLAC:focus, wavefront/donut analysis (also on DESI), RASICAM cloud camera Fermilab Institutional Review 2015

Optics UCL. UK ANL/FNAL CCDs, wafer from LBNL, packaged at FNAL Shutter, Germany Filter changer, Univ. of Michigan Filters, Japan Electronics,UIUC, Spain and FNAL Hexapod, Italy DECam Systems Imager, FNAL Observer interface and instrument controls (OSU, FNAL + others) Fermilab Institutional Review 2015

University and Fermilab Management roles in DECam Project leadership roles were distributed –Fermilab scientists and engineers covered management of 5 out of the 7 Level 2 subsystems –Level 2 managers for the online software from UIUC and OSU –Level 2 manager for the optics was from UCL MOUs and annual statement of work were used to document expectations and to transfer resources between Fermilab and universities We worked with each university to setup the relationship in a way that best fit their needs and those of the project based on their relationship with DOE, their base grant, their university and the DECam project. A few examples: –UIUC base grant included project funds that did not come to Fermilab, but we tracked with the rest of the project funds at Fermilab. –UMich had funding from their university, and their base grant and was augmented by DOE funding that came through Fermilab –OSU supported code developers initially with university funds and then with DOE funds that came through Fermilab Fermilab Institutional Review 2015

CCD work on DECam – example of a cross cut and coherence of the lab program At Fermilab we built up a CCD packaging and testing facility using the cleanrooms, infrastructure and technical expertise developed during the construction of the silicon vertex detectors for CDF, D0 and CMS Students and postdocs from universities visited Fermilab to gain hardware experience and help with the CCD testing The resources and expertise developed for DECam plus interactions with the astro-theory group led to directly to DAMIC and other projects mentioned in Juan’s talk CCD effort is continuing with DESI – Fermilab will package and test the red and near infra-red CCDs for the DESI spectrographs, and is bringing in students already Participating in LSST workshops on science with thick CCDs Fermilab Institutional Review 2015

DECAM Integration at Fermilab The integration and testing of all the systems together before shipping to Chile capitalized on Fermilab’s unique high bay facility and mechanical assembly expertise Mock observing runs brought in experts and experienced observers from around the collaboration and engaged them in the development of the complete system. Result: DECam worked “out of the box” in Chile Fermilab Institutional Review 2015

Telescope simulator at Fermilab with everything installed Fermilab Institutional Review 2015

DES Operations First light for DECam was Sept Now Fermilab manages DES Operations –Schedules all collaboration observers for DES nights (105/yr) –Travel support for US observers –Hardware and software support for DECam, coordination/funding for trips by experts within the collaboration –Storehouse of expertise in all camera systems, unique expertise for Cryo –Manages webpages, collaboration accounts, membership data base, speakers database –Fermilab Scientific staff (thy and exp) have leadership roles in 3 of the DES science working groups, including leading development of COSMOSIS tool to extract DE parameters University technical support on operations are funded through Fermilab –OSU, UMichigan, SLAC, BNL, APSU and UChicago Fermilab augments Data management effort at UIUC/NCSA –Critical support role enhances ability of collaboration to extract the DE science Fermilab Institutional Review 2015

DES Observers DES staffs the 105 DES nights (each season) with 2 or 3 observers depending on whether it is a Full night or Half night. This table shows the number of observing sessions by type of observer. The lengths are usually a week with some Run Manager’s serving 2+ weeks. Fermilab Institutional Review 2015 Year 1Year 2 International GS+PD 108 Int. Senior34 Lab GS+PD67 Lab Senior1211 US Univ. GS+PD811 US Univ. Senior45 CTIO24

DES observers Fermilab Institutional Review 2015

Scientific Computing Support for the DES Collaboration Fermilab contributions to Survey operations –Survey Strategy: Design and maintenance of the software that decides which exposure to take and Simulations of survey completeness for various strategies –Online database at CTIO, offline mirror at FNAL and associated web-based viewing tools Fermilab supports DES Data management –Provides tape storage for a disaster recovery backup of the reconstructed data from NCSA. –CPU cycles on FermiGrid for data processing (Year2 nightly processing and in the future, reprocessing for the annual release) –Major involvement in the redesign and reimplementation of the processing workflow Fermilab Institutional Review 2015

Scientific Computing Support for DES User community Support of interactive computing cluster DES uses the following Fermilab supported services/products –DocDB –Cern Virtual Machine File System for FermiGRID processing –Redmine for the internal experiment web pages –Electronic Control Room Logbook DES has the following custom tools developed by Fermilab –Speakers Bureau App – used to assign and track talks given at conferences –PubDB/Projects App – used for announcing science projects in DES and tracking their status through to the paper submission stage Leadership roles in a number of DES Science Working Groups Fermilab Institutional Review 2015

DECam Spin-offs engage the user community DECam ended with extra/spare engineering and science grade CCDs. These have been used for a variety of purposes: –DAMIC – detection of DM with CCDs. Developed into a full fledged collaboration joint with University of Chicago and other institutions, has best limit at low masses –Contributed to the upgrade of the detector in the LDSS3 instrument on the 6.5m Magellan II Telescope (Las Campanas Observatory, Chile). This included a leftover DECam CCD and electronics. LDSS3 is a wide-field (8.3 arcmin diameter) imager and spectrograph with multi-object capabilities. –Leftover small DECam CCDs loaned to universities and labs (Laird Thomson UIUC, LSST test at BNL, LSST test at SLAC/UCDavis, Barcelona for DES and PAU testing). These devices are great for testing new electronics and handling procedures without risking valuable science grade devices. Fermilab Institutional Review 2015

DES/Fermilab Connections to the broader DE community U. Chicago: –Fermilab Staff (Scott Dodelson, Josh Frieman, Steve Kent) advise graduate students on DES –Access to Fermilab cleanroom and CCD facilities –Now converting DECam lab space to CMB focal plane test and integration facility –Joint appointments Chicago-land DES workshops (ANL, FNAL, Chicago, UIUC) –Organized by DES early-career scientists (students and postdocs) –First one was at ANL, next one is at FNAL Feb. 23 Fermilab hosted a joint DES-LSST meeting –Very successful exchange of ideas and experience Fermilab is hosting the DESI annual collaboration meeting this year (May 2015) Fermilab Institutional Review 2015

DES is roughly half early-career (students, postdocs, non- tenured) scientists The careers of the early-career scientists is a collaboration wide concern. Early-career committee setup to specifically address concerns across the collaboration Speakers Bureau finds talks at workshops and conferences, tracks who is giving talks and aims to make sure the early-career scientists are getting important talks. Fermilab scientific computing built the Speakers database Organized sessions on a variety of career options (academic and non-academic) at the last two collaboration meetings DES has a newly formed Education and Public outreach group –Focuses on engaging the public in DES on many fronts (blogs, facebook, talks, etc) –Helps early-career scientists get exposure and experience in communication (turns out this is broadly useful!) Fermilab Institutional Review 2015

Fermilab DE program moving forward Scientific staff on DES are engaged in DES operations and science analysis –20 Fermilab Scientific Staff on DES span multiple divisions: 7-particle physics, 7-scientific computing, 3-astro-theory, 3 postdocs (thy+exp) –Analysis projects include constraints on clusters and using the overlap with SPT (one of the original motivations for DES), strong lensing, supernova, combined probes, etc (see Josh’s talk) Some are already transitioning to DESI and LSST, transition will continue –8 Fermilab scientists admitted to the DESI collaboration; key roles: BF DESI Project Scientist Gaston Gutierrez is leading the corrector barrel and top end efforts Juan Estrada is leading the CCD packaging and testing –4 Fermilab scientists are members of the LSST science collaboration (DESC); key roles: Scott Dodelson with support of the Fermilab scientific computing division is leading the development of a software framework for the LSST analysis, and COSMOSIS for extracting DE parameters CMB group is forming –2 Fermilab Scientists and one postdoc, will grow as project develops Fermilab Institutional Review 2015