Kosmos Review, 2-3 August 2010 OSU, Columbus, OH Kosmos Software NOAO Software Implementation Phil Daly

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH Overview What Is The NOCS? –How it works, history etc –KOSMOS implementation –Data management Schedule & Deliverables Q & A

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH What Is The NOCS? The NOAO Observation Control System (NOCS) was developed for NEWFIRM, a 16 megapixel IR mosaic camera. The architecture diagram for NEWFIRM is shown to the right and the NOCS. Inherent in the NOCS paradigm is: scripting control of various sub-systems: – data handling system – MONSOON detector controller – Telescope system motion(s) – instrument hardware – meta-data repositories NGUI, a separate utility, provides a scripting engine for creating well- defined science recipes.

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH Re-Using The NOCS Clearly it makes sense to re-use code where applicable and not re-invent the wheel. The rationale for using the NOCS is:  It has 3+ years of use – First Light: NEWFIRM, KPNO Mayall 4m, February 2007 – First Light: NEWFIRM, CTIO Blanco 4m, May 2010 – First Light: Mosaic 1.1, KPNO Mayall 4m, scheduled for October 2010 – First Light: Mosaic 1.1, KPNO WIYN 0.9m, anticipated in semester 2011A – First Light: KOSMOS, KPNO Mayall 4m, planning for semester 2011B – First Light: COSMOS., CTIO Blanco 4m, planning for semester 2012A?  Failure modes are well documented  It is ideal for rapid deployment projects (as recommended by the ReSTAR report) since 80-90% of functionality exists  It is written with the GWC infrastructure software, common to all 3 telescopes, built-in  It is integrated into the NOAO E2E data system  It has been commissioned on 2 of the 3 telescopes  Support staff at distributed sites are now familiar with it  It will offer a “common look and feel” to NOAO instruments

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH The Future The table to the right shows current and planned instruments for the NOAO operated science system. As can be seen, these facilities represent different science capabilities on different telescopes in different wavelength regimes. Rather than implement a complete new control system for each instrument, we undertook a modest design study to see if the NOCS could be re-used to tackle these other instruments. So far, we have the NOCS working on the 4m telescopes (for NEWFIRM) and are well on the way to having Mosaic 1.1 using the NOCS too. KPNO Mayall 4m NEWFIRM 4k x 4k IR Imager Now Mosaic 1.1 8k x 8k OUV CCD Imager KOSMOS 2k x 4k OUV CCD Spectrograph CTIO Blanco 4m NEWFIRM 4k x 4k IR Imager Now COSMOS 2k x 4k OUV CCD Spectrograph KPNO WIYN 0.9m Mosaic 1.1 8k x 8k OUV CCD Imager

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH The NOCS Pre-Requisites Original design in 2002 (?) NOAO Observation Control System had to conform to –GWC in 4m environment –GPX over sockets –Tcl/tk widely used at KPNO –Software trade study implied DRAMA as best message passing mechanism –Common API for data transport –Not burden KPNO support staff All done in Tcl/tk with loadable shared libraries—buzzword free zone (no Java, XML, Web 2.0, python etc etc)!

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH The NOCS Interfaces

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH The K.I.S.S. Principle wc -l *.tcl –Includes comments and blank lines –Includes infrastructure code NOAO Observation Control System –NMSL, talks to Monsoon/Torrent 1792 lines of Tcl/tk (Mosaic 1.1) –NTCS, talks to the telescope 1090 lines of Tcl/tk (Mosaic 1.1) –NICS, talks to instrument hardware 1343 lines of Tcl/tk (Mosaic 1.1) –NOHS, talks to DHS and listens for environment meta-data 790 libnes of Tcl/tk (Mosaic 1.1) –NGUI, prepares scripts for observation 3045 lines of Tcl/tk (Mosaic 1.1) –nocsCommon.tcl, 1061 lines of Tcl/tk (Mosaic 1.1)

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH NOCS Features NOCS is elegantly simple NOCS has modular and orthogonal tasks –Not a monolithic software system –Change any one does not affect the others –Task may be run independently for test NOCS has complete logging for PM debugging and “on sky” wellness checks Create scripts and run them separately! Scripts tell the GUIs what to do Supports test, for example: –testnics filter U –testntcs offset “30 45” –Uses same comms path as scripts

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH Message Sequence Chart

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH A Very Simple Script #!/bin/sh ditscmd nmsl nmsl_gpxSetAVP Argument1="intTime=0.7" ditscmd nmsl nmsl_gpxSetAVP Argument1="rowBin=1" ditscmd nmsl nmsl_gpxSetAVP Argument1="colBin=1" ditscmd nics nics_filter Argument1="1" EXPID=`$NEWFIRM_BIN/msd` ditscmd nohs nohs_newobs Argument1=“NOCID=${EXPID}\…” ditscmd nmsl nmsl_gpxSetAVP Argument1="expID=${EXPID}" ditscmd nmsl nmsl_gpxGetState Argument1="IGNORE" ditscmd nmsl nmsl_gpxStartExp ditscmd nohs nohs_endobs

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH Scripts: Can Do / Can’t Do Can –Can abort single observation from NMSL (with cleanup) –Abort script using Ctrl/C Then follow instructions for cleanup –Re-use them (or use iterators from NGUI) –Edit them (but not advised) –Create hierarchies (but not advised with exceptions) –Be as long/complex as you like Have seen an line script Mark Dickinson linearity test:~ 250 observations, 3.5 hours Can’t –Automatically clean up after a Ctrl/C –Re-start from specific place (but you can use vi)

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH KOSMOS Implementation X X X XX X

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH NMSL NMSL - Talks to MONSOON/Torrent if voltages and temperatures are nominal ICD nothing else to do?

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH NTCS NTCS - Talks to Telescope etc ICD Add rotator, secondary focus

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH NOHS NOHS - Listens to 4m env, talks to DHS ICDs 6.1, Create nohs.gwc file (already done in appendix A, ICD 6.1)

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH NICS NICS - Talks to hardware via NICC ICD 3.1, Add KOSMOS elements, create filters file

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH nics.cfg #+ # NEWFIRM Filter Wheel List 06-Jan-2010 for semester 2010A # #NICC \NGUI \Combination \FW1 Pos\FW2 Pos\Serial Number(s) #- J \J \JX + Open \1 \8 \2104+open H \H \HX + Open \2 \8 \3104+open Ks \Ks \KXs + Open \3 \8 \4103+open 2124 \2124 \2124/68 block nm \4 \4 \ \Br gamma \2124/68 block nm \4 \5 \ J1 \J1 \J1 blocker + J1 \5 \1 \ J2 \J2 \J2+3 blocker + J2 \6 \2 \ J3 \J3 \J2+3 blocker + J3 \6 \3 \ H1 \H1 \H1+2 blocker + H1 \7 \6 \ H2 \H2 \H1+2 blocker + H2 \7 \7 \ Dark \Dark \DarkSlide nm \8 \4 \dark+4301

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH NGUI NGUI - Creates scientifically useful scripts from OCDD input Not an observing tool!

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH NGUI

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH NGUI / xterm

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH Data Management

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH Data Handling System Collects data and meta-data from PAN and NOCS, processes data (as required) and assembles final FITS file on disk ICD 6.1 provides structured comms and procedures for data capture regardless of instrument Data processing independent of readout cadence so processing continues as long as there is data waiting

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH Data Handling System Supervisor layer controls actions in the DHS providing single point of contact for instrument (excluding pixel data capture) Distributed application using component architecture Components may be added, as needed, for multiple PANs Produces pipeline ready product Post-processing for ingestion into NOAO science archive via Save the Bits

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH DHS Components Supervisor –Primary GUI interface –Provides user feedback and control Collector(s) –Responsible for reading data from PAN and NOCS into shared memory cache SMCMgr –Rectifies pixel orientation (lower left corner) –Provides RTD of sub-rasters

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH DHS Components PXF –Puts segments from SMC onto message bus DCA (data capture agent) –Spawns the Keyword Translation Module (KTM) to transform PAN/NOCS metadata into standard FITS header, provides rough WCS, science keywords, etc –Assembles final FITS image SMC (shared memory cache) –Interface to manage the many meta/data segments –Allows for data to be managed in memory –Maintains relationship of associated segments –Provides buffer for data to backup during fast sequences

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH DHS Supervisor GUI

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH NOCS Schedule

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH NOAO Staff Use people who know the system and developed original system where possible …

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH Hardware Rack 4 x Lancelot 1184-T, 1Tb disk, $6980 (1 spare) 2 x Systrans PCIe, $7280 (1 spare) 1 x 12U rack, $500 1 x HP ProCurve G Gigabit switch, $360 1 x TrendNet TK-802R KVM switch + cables, $172 1 x 16-port network power controller, $295 Total: ~$15,500 + tax + S/H

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH Deliverables From OSU: iebUtils library by 31-Oct To OSU: completely populated rack (hardware and software) for OSU to perform lab integration and test using a full NOCS system by 31-Jan-2011 Fallback: mec etc

Kosmos Review, 2-3 August 2010 OSU, Columbus, OH The End … ? KPNO Mayall 4m NEWFIRM 4k x 4k IR Imager Now Mosaic 1.1 8k x 8k OUV CCD Imager KOSMOS 2k x 4k OUV CCD Spectrograph CTIO Blanco 4m NEWFIRM 4k x 4k IR Imager Now COSMOS 2k x 4k OUV CCD Spectrograph KPNO WIYN 0.9m Mosaic 1.1 8k x 8k OUV CCD Imager