MWA Operations Management Plan (OMP, v. 3) Ron Remillard (MIT); MWA Project Meeting, June 6, 2011

Goals for OMP Completion Explain OMP v. 3 at Dec. project meeting (current session)  Available now on MWA twiki  …or browse Gather feedback from team and Board (this week) Produce OMP v. 4 (by 2012 Jan 6) Board Review of v. 4 during January telecon Produce OMP v. 5 by 2012 Jan 27 Use OMP v. 5 as a basis for planning in 2012  Support proposals for Operations funding  Help organize activities of Science Council  Help organize observing proposals from Science Package teams Review OMP at June 2012 project meeting

Perspectives on the MWA Operations Management Plan, v.3 Outline MWA Activities as a Managed Observatory (not shoestring theater; one can trim, if desired or necessary) Organized in categories: Facilities support; Observations support Primary Elements of OMP  Time-line of events that frame MWA observations  Outline assumptions about MWA project support external to Operations  Specify Operations staffing and work tasks  Specify support required for maintenance and repair  Sort requirements into Operations costs and Other Resources (i.e., faculty, science grants, fellowship postdocs, students, etc.) Strategy: Make some choices and seek feedback

Time Line of Events for MWA Operations (slide 1) (Science) Science Council issues call for proposals, per MWA Board Time Allocation policies. (Science) Observing proposals are submitted to the Science Council by Science Package leaders and other sources consistent with the proposal call. (Science) Science Council oversees the proposal review and issues a target list containing target name, position, exposure goal, observing mode, and observing constraints. (Target lists should be issued no less frequently than every 6 months, and delivery should be at least one month before the first observation.) (Science) Requests for additional Science Council targets or Targets of Opportunity (ToOs) from Director’s discretionary time, may be issued, in accordance with MWA Board policies. (Operations) Observing schedule is made from the approved target lists. (Operations) Observing commands are generated to implement the schedule. (Operations) Supplemental observing commands are generated for routine observing modes that support instrument health and safety, with procedures TBD. (Operations) The command load is stored in the M&C command database to conduct the observations, under the supervision of the Duty Scientist. (Operations) Target revisions and ToOs are accommodated, as resources allow.

Time Line of Events for MWA Operations (slide 2) (Operations) Observations are conducted, generating correlator data, RTS output images, RTS calibration data, and M&C database records, with contents and formats defined in MWA standard data products [9]. (Operations) Correlator and RTS data products are transferred via the fiber link from MRO disks to the primary MWA Archive at the Pawsey Centre in Perth. (Operations) MWA data transfer software monitors the data transfer status and activates the backup recording system, as needed. (Operations) Upon completion of the observations, the associated entries in the M&C database are transferred to the MWA Archive. (Pawsey Center) All MWA data products from MRO are stored in the MWA Archive, under the supervision of the MWA Data Archive Manager. (Science) A second version of the RTS operates in Curtin (details TBD) to support RTS code revisions, algorithm testing, and reprocessing computations for selected data, using the MWA archive.

Time Line of Events for MWA Operations (slide 3) (Science) Data is distributed from the primary MWA Archive to secondary Archives at MIT and Melbourne. (Pawsey Center) Data is distributed from the primary MWA Archive to general science users, per MWA Board policies. (Science) Scientific analyses are conducted, leading to publications of results. (Operations) Monthly (TBR) reports regarding targets and good exposure time are prepared by the Duty Scientist for project evaluation. (Science) Instrument performance tests and trend analyses are performed on MWA data for project evaluation, under the direction of the Operations Director.

Operations Staff positionFTElocation fundingOps FTE Operations Director0.5Curtin Faculty -- Site Manager0.5Curtin Ops budget 0.5 Operations Engineer1.0Curtin Ops budget 1.0 Assistant Engineer1.0Curtin Ops budget 1.0 Data & Archive Mgr.0.5Curtin Ops budget 0.5 Duty Scientist1.0TBD Ops budget 1.0 M&C Software Engineer0.5TBD Ops budget 0.5 RTS Software Engineer1.0TBD SupSciFellow -- Computer/Network Mgr.1.0TBD Ops budget 1.0 ____ ____ total

Operations Staff OPERATIONS DIRECTOR (0.5 FTE; Curtin Faculty) Oversee OMP implementation. Oversee Operations budget. Report Operations status, trends, costs, risks, and changes to the MWA Board. Expedite hardware repairs, working with the Operations Engineer. Tap MWA Heritage expertise from partner institutions, as needed. Find Project resources to tackle special problems, as needed. Oversee high-level monitor and trend analyses to guarantee Observatory health & productivity. SITE MANAGER (0.5 FTE; Operations budget) Oversee the working conditions of the MWA central building and housing facilities and work with CSIRO to resolve problems. Perform scheduled maintenance inspections of MWA Tiles and Receivers. Perform resets of hardware or utilities that cannot be achieved with M&C. Assist efforts of Duty Scientist or Operations Engineer, as needed.

Operations Staff OPERATIONS ENGINEER (1.0 FTE; Operations budget) Diagnose hardware problems and make decisions on repairs and replacements. Oversee procurement and site deliveries for hardware repairs and replacements. Maintain RFI compliance for MWA hardware at MRO, per CSIRO standards. Conduct laboratory tests needed to understand failures. Implement hardware changes that are cost effective and increase productivity. Recommend changes to M&C, as related to hardware monitoring. ASSISTANT ENGINEER (1.0 FTE; Operations budget) Assist in all of the work tasks of the Operations Engineer.

Operations Staff DUTY SCIENTIST (1.0 FTE; Operations budget) Make the M&C command schedule from the monthly target list. Troubleshoot M&C failures and resume observations. Recommend changes to M&C software or the user interface. Respond to requested schedule changes and ToOs, as resources allow. Track good exposure times for targets and issue reports monthly (TBR) for evaluation. M&C SOFTWARE ENGINEER (0.5 FTE; Operations budget) Consult with the Duty Scientist and Operations Engineer to track M&C performance. Revise M&C software to fix problems. Upgrade M&C to improve designs, interfaces, capabilities, robustness, or efficiency. Recompile, test, and install new M&C versions, as needed. Provide backup support to the Duty Scientist to continue scheduled observations.

Operations Staff RTS SOFTWARE ENGINEER (1.0 FTE; Super Science Fellow) Maintain RTS operation to produce calibrated MWA images in real time. Characterize statistical properties of deep images, compared to statistical expectations. Organize efforts to compare RTS methods to standard analyses, using selected observations. Revise RTS algorithms and install new RTS versions, as needed. Provide assistance for iterations of RTS processing via the MWA archive, as needed. COMPUTER & NETWORK MANAGER (1.0 FTE; Operations budget) Maintain proper operation of computers and receiver SBCs operating at MRO. Maintain MWA work stations associated with the MWA Archive. Maintain MWA networks at MRO and in Perth and handle security issues. Upgrade operating systems when necessary. Facilitate network connectivity for MWA team to MRO and the MWA Archive. DATA & ARCHIVE MANAGER (0.5 FTE; Operations budget) Track and ensure successful archiving of MWA data products. Facilitate the efforts of MWA team to retrieve MWA data from the Archive. Assist the Director in using the archive to track instrument performance and trends.

Operations Costs Category Number year 1 (kAUD) Operations Staff ? Diesel fuel 30/hr 250. ? Transportation 24 trips 50. Boolardy Housing 180 x 2 people50. Hardware repair beyond spares50. ? __________ _____1M Contingency ?? Overlooked ??

Final Comments Loose ends:  Board approval of Time Allocation Policy  Definition of Operations start e.g., approvals?: Project Scientist (Commissioning complete) Project Director (Project is Ready for Operatons) Operations Engineer (Hardware is Ready for Operations)  Consistency of PMP and OMP Define roles (e.g., monitoring) for users (require in time allocation policies?) Define Engineering observatins and cycles that need to be added to science observation. Send feedback and recommendations Let’s get this finished!