1. KECK INTERFEROMETER KISWG 10/29/20031 ORR Readiness Mark Colavita & Peter Wizinowich KISWG Pasadena 10/29/2003

2. KECK INTERFEROMETER KISWG 10/29/20032 Question to Address (2/3/2003 pre-ORR presentation) At the ORR –Will the instrument be capable of achieving the requirements in the PLRA at the start of routine science operations? (V 2 mode) –What are the relevant performance metrics and current performance? (V 2 mode) Today –What are the specific performance requirements, and how do they relate to the PLRA? –How will compliance be demonstrated? –Are they technically achievable? –What is the current performance? –What is the plan to get from current performance to ORR? Is the plan realistic? –What are the risks for routine science operation? How will they be mitigated? Are there additional risks not yet identified that need to be addressed?

3. KECK INTERFEROMETER KISWG 10/29/20033 Specific Performance Requirements (2/3/2003 pre-ORR presentation) PLRA * requirements for V2 –System Performance Goals »Provide 5% rms accuracy V 2 amplitude measurements for targets as faint as K=9 using the two Keck telescopes »Provide at least 90% uptime for the interferometer and 90% uptime for the two 10m telescopes and their AO systems –System Performance Requirement »Provide at least 90% uptime for the interferometer and 80% uptime for the two 10m telescopes and their AO systems * Program Level Requirements Appendix to Navigator Program Plan

4. KECK INTERFEROMETER KISWG 10/29/20034 Pre-ORR Testing from Oct Observing Run Targets included measurements of well-characterized PTI binary –Used neutral density filters to attenuate this bright star to simulate K=9 ORR requirements »Data accuracy still being assessed »Observation K=9 spec will need to be qualified with a V or R magnitude, as stars faint to AO are less well corrected, and provide less flux into fringe tracker –(Fringe tracker measures flux * Strehl) Plan to provide this for ORR –Also tests of fringe tracker low-V2 tracking limit

5. KECK INTERFEROMETER KISWG 10/29/20035 Oct Data (Most processing still in-progress)

6. KECK INTERFEROMETER KISWG 10/29/20036 Prior (May, Jan) accuracy tests

7. KECK INTERFEROMETER KISWG 10/29/20037 Other tests

8. KECK INTERFEROMETER KISWG 10/29/20038 Provide at least 90% uptime for the interferometer and 90% uptime for the two 10m telescopes and their AO systems (2/3/2003 pre-ORR presentation) Quoted directly from the PLRA –NB: not a measure of “open shutter time” or efficiency, which is actually quite low for any instrument, like IF, that is calibration-limited –A measure equal to 100% minus the downtime required to fix IF hardware or software problems during scheduled science observing Telescopes/AO addressed in a few slides Demonstrating compliance –Mostly proper bookkeeping »Identify a timekeeper to track system state vs. time »Demonstrate compliance on two runs prior to ORR with good weather and median or better seeing

9. KECK INTERFEROMETER KISWG 10/29/20039 System uptime & scans per hour

10. KECK INTERFEROMETER KISWG 10/29/200310 Uptime summary Interferometer uptime  90% 2/5 runs Facility uptime  90% 1/5 runs;  80% 3/5 runs October run problem areas –Major »Telescope control system crashes K2 continued to have problems after the IF run supporting DEIMOS Problem may be resolved – need to test –Minor – most in process of being addressed »Facility AO offload problem (possible interaction from IF angle tracker) K2 AO Wavefront controller crashes »Interferometer Sequencer crashes Aligner crate reboots Telemetry server memory growth – solved Goal is to achieve ORR metric performance on next two runs

11. KECK INTERFEROMETER KISWG 10/29/200311 Performance, functionality, efficiency goals (2/3/2003 pre-ORR presentation) These follow from the desire to maximize the science achievable at the specified performance level

12. KECK INTERFEROMETER KISWG 10/29/200312 Performance, functionality, efficiency goals, 1 (2/3/2003 pre-ORR presentation)

13. KECK INTERFEROMETER KISWG 10/29/200313 System uptime & scans per hour

14. KECK INTERFEROMETER KISWG 10/29/200314 Scans per hour 86 science plus engineering scans in 22.5 clear hours Peak scans on each of 4 nights: –2, 3, 6, 8 Sustained high throughput impacted by –System faults –Ease of observing target »Faint targets, requiring optimization of AO, KAT, FATCAT, or which end up not being trackable, count against scans per hour »Weather (light to moderate cirrus, which doesn’t count against weather loss) –May need to qualify our 6 scan/hr goal to apply to sources not pushing system limits Night 3 of Oct run Peak scans/hr for this night

15. KECK INTERFEROMETER KISWG 10/29/200315 Performance, functionality, efficiency goals, 1 updates to 2/3/2003 pre-ORR presentation

16. KECK INTERFEROMETER KISWG 10/29/200316 Performance, functionality, efficiency goals, 2

17. KECK INTERFEROMETER KISWG 10/29/200317 Performance, functionality, efficiency goals, 3

18. KECK INTERFEROMETER KISWG 10/29/200318 Procedural Requirements for the ORR From the PLRA: “Provide an operational infrastructure including trained staff, facilities, hardware and software, so that the Keck Interferometer will operate as a facility for use by the community scientists.”

19. KECK INTERFEROMETER KISWG 10/29/200319 Procedural Requirements for the ORR From LaPiana: “Need (A) requirements which you can track against your progress to actually achieve the metric performances, and the (B) ability to do it routinely and having an organization in place that is well trained to do so with robust and stable software (at WMKO & MSC).” Questions to address at the ORR from LaPiana: 1.Will the ground data system (at WMKO and at MSC) implementation and testing be complete prior to the start of routine science operations? 2.Will the required operations plans and procedures, including those for anomaly responses, be completed and tested prior to start of routine science operations? 3.Are the operations organization, staff and facilities in place (WMKO and MSC)? 4.Will the necessary training activities be completed prior to start of routine science operations? 5.Are the necessary set of commands and sequences ready to support nominal operations? Will all action items, liens, etc., be satisfactorily dispositioned? 6.Are the risks well understood and acceptable for routine science operations?

20. KECK INTERFEROMETER KISWG 10/29/200320 1. Ground Data System Completion Will the ground data system (at WMKO and at MSC) implementation and testing be complete prior to the start of routine science operations? At the pre-ORR: The primary demonstration will be successful CARA operation of the interferometer for V2 science. We will track this performance over the next ~ 7 runs. The last 3-runs prior to the ORR will be performed at the full ORR requirement level. CARA personnel have successfully operated the interferometer for most of these runs. We have tracked performance during these runs. Many, but not all, of performance numbers demonstrated in Oct. run. At most can demonstrate full ORR-level compliance during only 2 runs (Nov. & Jan.) prior to ORR.

21. KECK INTERFEROMETER KISWG 10/29/200321 2. Operations Plans & Procedures Will the required operations plans and procedures, including those for anomaly responses, be completed and tested prior to start of routine science operations? Operations procedures for daytime checkout & night-time operation are in place. Anomaly responses need completion Including spares & troubleshooting training Configuration management (software, opto-mechanical & electronics) need to be defined & agreed upon Especially important during ongoing development Major software upgrade will happen with RTC 2.1 Also need upgrade plan (plan to implement upgrades without compromising operational system). Would be easier if some subsystems had already been handed over to CARA Need to approve/implement plan for maintaining cleanliness of optics/basement

22. KECK INTERFEROMETER KISWG 10/29/200322 2. Operations Plans & Procedures Current operations procedures: 1.Punchlist with new priorities distributed after each run. Reviewed bi- weekly. 2.Schedule for summit activities maintained. Reviewed weekly. Incorporated in Observatory scheduling. 3.Daytime summit activities entered into IFDEV & Observatory daylog. 4.Observing support personnel (daytime, nighttime & on-call) list distributed prior to run. 5.Setup & observing procedures available on-line (next slide). 6.Afternoon pre-observing checklist/procedure performed. 7.Plan for night’s observing distributed. 8.Observing log maintained & published to telescope nightlog. 9.Post-run summary distributed, including priorities for next run. 10.Punchlist with old priorities closed out. 11.Post-run review held with Observatory/Project management.

23. KECK INTERFEROMETER KISWG 10/29/200323 Operations Procedures

24. KECK INTERFEROMETER KISWG 10/29/200324 3&4. Operations Organization & Training Are the operations organization, staff and facilities in place (WMKO and MSC)? Will the necessary training activities be completed prior to start of routine science operations? Organization John Gathright has assumed Operations Coordinator role Responsible for coordinating/scheduling/staffing activities in support of observing runs, observing prep & daytime experiments 2 (of 3) technicians report directly to him

25. KECK INTERFEROMETER KISWG 10/29/200325 3&4. Operations Organization & Training Facilities Control rooms set up at headquarters & summit Control rooms routinely used for interferometer operation Instrument performance capabilities need to be made available to the community for observing planning

26. KECK INTERFEROMETER KISWG 10/29/200326 3&4. Operations Organization & Training Staff 3 Interferometer specialists trained in operation of Interferometer for daytime alignments, checkouts & tests, & night-time operations 2 technicians trained in daytime alignments & checkouts Plan to hire an interferometer operator & an interferometer specialist to relieve operational load on above personnel Software engineers (2) able to support day & night-time troubleshooting. Electronics engineer able to support daytime troubleshooting. Mechanical engineer gaining some instrument familiarity.

27. KECK INTERFEROMETER KISWG 10/29/200327 3&4. Operations Organization & Training Staff Currently 1 JPL & 1 MSC person participates in each run. Can we fill their roles after the ORR? JPL person really helps with troubleshooting We are gaining more familiarity with instrument & resolving problems (JPL still on-call) MSC person really understands science priorities We need more insight &/or MSC on-call

28. KECK INTERFEROMETER KISWG 10/29/200328 5. Commands & Sequences Are the necessary set of commands and sequences ready to support nominal operations? Will all action items, liens, etc., be satisfactorily dispositioned? Alignment Procedures exist and are used. All mirrors and shutters needed are remotely controlled. Pre-Observing checklist Checklist exists and routinely used by CARA personnel. Observing procedure/checklist CARA personnel routinely operate. MSC tools procedures CARA personnel recently operating. Post-Observing Tapes written and shipped.

29. KECK INTERFEROMETER KISWG 10/29/200329 5. Commands & Sequences Target acquisition (Manual & Automated steps) 1.Interferometer Sequencer (IS) operator selects target from list IS loads parameters & initiates Telescope Sequencer (TS) IS configures FDLs for observation 2.TS initiates the following (can change for repeat observation): Telescope control system slews, then tracks, the telescopes Auto-acquisition tool positions brightest target on acquisition camera for AO system AO auto-setting tool optimizes AO parameters on target AO operator reports AO loops locked to IS operator 3.AO system offloads tip/tilt and focus to the telescope 4.IS operator closes Angle Tracker (KAT) loop: KAT loop automatically offloads IF tip/tilt mirror to AO wavefront sensor offsets AO operator unloads offsets to field steering mirrors if > threshold 5.IS operator initiates IS to acquire fringes IS scans to peak signal on fringe tracker IS searches for fringes

30. KECK INTERFEROMETER KISWG 10/29/200330 5. Commands & Sequences Remaining issues: Automated night-time LDL repositioning (manual & too slow) This is currently a high priority Automated daytime alignment (labor & telescope intensive) Likely a lien at the ORR User interface is complex & distributed (can make mistakes) Plan to implement a simpler quick-look Better version will come after RTC 2.1 upgrade Observing sequence automation should be improved to achieve scans/hour with reduced human interaction Selectable level of telescope sequencer operation Offloading wavefront sensor centroids to FSMs Reuse of AO parameters Remote configuration A few remaining hands-on operations

31. KECK INTERFEROMETER KISWG 10/29/200331 6. Risks Are the risks well understood and acceptable for routine science operations? Risks identified at ORR requirements review: Weather, wind shake and seeing There are nights when IF observing simply cannot be done productively or when only bright targets can be used. Best choice in some conditions would be to switch to another instrument program that can succeed with poor seeing. Hardware failures Mitigation: List of critical spares and backup approaches to be identified & implemented prior to ORR.

32. KECK INTERFEROMETER KISWG 10/29/200332 6. Risks Development during operations Mitigation: Configuration control. Observatory-wide software configuration plan currently being approved. Will apply to Interferometer & AO ops. Inteng & tag of intdev nearly in place to protect software. ECR process in place for AO opto-mechanics. Mitigation: Careful coordination of development & operations activities (using current operations procedures). Number of shared risk & TAC allocated V2 science nights Too many nights could result in inadequate CARA involvement in development &/or inadequate access to interferometer for development. Possible mitigation: Limit the number of science nights. Additional operations staff. New: Load on CARA staff if system requires lots of hand-holding

33. KECK INTERFEROMETER KISWG 10/29/200333 ORR readiness summary Will probably be OK on V 2 accuracy spec – need to complete analysis of Oct data Uptime still problematic –Working on major (and minor) problems from Oct –With satisfactory resolution of major, and several of the minor problems, should be possible to achieve in November & Jan. Scans per hour still low, although brief good peak performance –Some issues with brightness of target – will not apply if target requires extensive optimization –Improvements to system uptime and user interface should improve performance OK on most other areas, except for LDL automation, which is a current focus