1. New Mount Controller Software (the 30,000 foot view) "The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends upon the unreasonable man." -- G. B. Shaw D. Clark 10/21/14

2. Project Goals Replace ancient computer hardware and software with a solution that is: – Modular – Maintainable – Robust – Leverages new technologies – Uses appropriate “best practices” – Replaces special software and drivers with an open-source solution

3. Modular Software Mount control software is broken down into key components that are assigned to specific people. Key components tend to have one or only a few functions. Components communicate with each other and the outside world using a common framework. Functions are decoupled using messages across the framework to enable distributed operations.

4. Maintainable Software Avoid use of special hardware and software libraries. Use standard Linux libraries, particularly thread-safe POSIX routines in time-critical code. Use “off-the-shelf” software components, e.g. redis, AstroPy, SOFA, Aladin, among others as examples. Enforce language compatibility among the development team (C, Python, Node.js, Javascript, others?). All software is maintained in a git repository, and the build environment is kept generic and not specific to one developer’s particular machine and toolchain setup. Regular meetings are held among the development team to share progress and information about how their software works.

5. Robust Software The use of a stable CentOS kernel and building code only against that kernel library provides general insurance against kernel and security updates breaking functionality. A policy of using 3 rd -party libraries and software packages offloads bugfixes and maintenance from the MMTO staff (so long as packages are updated as necessary). General software maintenance given commonality in the core languages makes it possible for more than one person to work on the code. An effort to document the software as the project progresses will pay dividends in the future as the Observatory’s needs evolve.

6. Leveraging New Technologies Computer languages and software packages evolve rapidly and MMTO should not depend on a particular set of tools and algorithms necessarily being forever static – within reason! HTML5, Node.js, redis, and other software that may become available in the future will position us well for future improvements and upgrades as needed. The mount control hardware, therefore, needs to be designed around the concept of flexibility (e.g. NO special hardware and a generic PC). Hence, the choice of EtherCAT and ethernet for i/o to decouple the computer hardware from the drive system electronics.

7. Use “Best Practices” A git repository for archiving the software. Modern browser-based interfaces as the GUI front-end to the system. Use an up-to-date computer and Linux kernel. Do avoid “bleeding edge” Linux distros: go with a stable one and plan to upgrade only with major releases (e.g. CentOS 7  CentOS 8) on roughly 5-year increments. Use standard string-messaging protocols (e.g. JSON) and avoid strange libraries and non-standard message dictionaries. Leverage high-performance NoSQL software as a fast data framework, so that parts of the code could be distributed to machine(s) other than the mount PC. Modularized code sections can and should be individually tested before deployment.

8. Mount Control I/O EtherCAT MasterHeidenhain EIB741 Library Dedicated Ethernet Port MMTO Ethernet Drive amplifiers, etc. Axis Encoders

9. Software Components D:\Shuttle Documents\mount_software_breakdown.vsd Not all software shown here necessarily runs on the mount computer; network sockets and the message backbone make it possible to distribute the work.

10. Software Component Details Time Service Uses the standard Linux NTP service along with the MMT GPS time server and outside network services (IERS) to get the current DUT (+ polar motion), and outputs UT1 and TT values accurate to milli-seconds, adjusting the mount computer clock on a regular basis. This proper date/time is consumed by astronomy computations and other “customers”. Astronomy Computations Uses the standard libraries (SOFA) to compute target positions to point the mount to the observed star position. No distinction should be made between sidereal and non-sidereal targets for uniformity of the code paths (e.g. use SOFA’s UT1 and TT time objects). TPOINT Corrections The necessary pointing corrections from the astronomy computation should be applied with this code to separate out the introduction of pointing-fit corrections to the computed target position. Encoder and Telescope Corrections Here we add whatever offsets or other effects from tube flexure and hysteresis that aren’t covered by TPOINT corrections. We may choose to implement a LUT or polynomial correction factor to the encoder position data, for example. Focal Plane Adjustments Provides adjustments to the demanded target position to align the telescope to a given point in the image plane (e.g. feed an off-axis imager). Apply Limits In this portion of the code, the computed mount position should be checked against the motion limits and only if they can be met will the position be sent to the mount servo loops. The “time to limit” values for each axis can be computed here as well.

11. Catalog Interface Provide users with a way to browse and pick a target star from a standard catalog, or use a catalog provided by the observer(s) and send the chosen coordinates to the astronomy code. User Interface Much like the existing GUI, provide a way to turn drives on and off, enter fixed positions for the mount, and other necessary functions for operation. Safety Interlocks Communicates with the interlock hardware and responds to either an interlock event (killing a drive axis), or a user interaction (turning on the drives). The interlock VME computer will eventually be replaced with an EtherCAT interface currently in the hardware-design pipeline. Servo Loops Code that actually uses the EtherCAT and EIB741 interfaces to the drive hardware and controls the telescope position. Pointing Offsets Provide a socket-based mechanism for pointing offsets to be introduced from other “customers” of the telescope (example: instrument offsets sent by a separate computer). Autoguiding Inputs Another socket-based mechanism for accepting tracking correction demands from autoguiders. Paddle Inputs Support manual guiding corrections over a different network socket. Event Logging Service to provide notice of important changes to the mount system, such as drives being enabled, etc. Publish/Subscribe Interface Service to expose variables within the mount control system to other consumers of the data via the standard publish/subscribe methods within redis (or others – SAO messages?). Telemetry Service Code that continuously collects system performance data, and writes collections of the data to a database. The servo performance data collection will be very similar to the existing system, only with much higher granularity since all the loop data from all axes will be continuously available.

12. Existing Software Inventory Of 56 source C files in the mount git repository, about 5 can be ported to the newer software infrastructure almost as-is. These are the interlock computer interfaces, which cannot be changed until the interlock computer itself is replaced. Of the remaining 51 files, 25 are completely removed. Now only about half the existing code base needs to be rewritten, and large portions of that (e.g. servo code) are auto-generated. Many can be greatly simplified in the process. Parts of the code (e.g. EtherCAT and EIB741 interfaces) are already available, or can quickly be produced (e.g. time service mostly uses existing Linux NTP service plus the IERS query server we already have – with minor modifications). All of the Xephem software in the /libastro source directory can be replaced with SOFA libraries; SOFA can compute sidereal and solar- system target coordinates through use of UT1 and TT time objects.

13. Mount Source File List

14. First Steps We will build up a complete single-axis controller for the rotator axis for use in day-time testing and software validation. This computer and hardware: Uses a reclaimed Shuttle box running CentOS 7 patched to run an rtpreempt kernel. Has the etherlab.org EtherCAT master stack installed. Will be attached to the interlock system and the rotator axis via the appropriate EtherCAT terminals, already present in the interlock cabinet. Still need to either adapt the existing 11uA output heads to 1Vpp signals, or buy new encoder heads to attach the rotator tape encoder to the Heidenhain EIB741. Alternatively, we have in hand incremental encoder input terminals for EtherCAT and can use the existing interpolator box outputs. Using the new computer and EtherCAT, we will be able to easily switch over the system cables and run the new equipment in parallel with the existing mount control system to perform testing as needed. The computer will need a minimal set of the code base necessary to run as an alternate axis controller: Time Service Astronomy Code to compute needed rotator positions Servo code and logic Interlock interface User interface Telemetry In this way, the needed system software can be “test flown” without disturbing the existing mount control system.

15. Next Stage Partially in parallel with the first stage, application development can be start with moving the mount software over to the new system: 1.Port the interlock computer interface to the new computer – prepare for the eventual replacement of the interlock VME crate. 2.Develop and test EtherCAT-based encoder readout, and/or work on moving to the proposed new Heidenhain RCN8500 units (which cannot be installed in the existing mount system as-is). 3.Get the telemetry and publish/subscribe interfaces ready to run. 4.Complete development of new servo loops for the main telescope axes. 5.Add in the autoguiding and paddle network support services. 6.Prepare hardware for day-time testing of new main-axis controls. 7.Finally, complete new user GUIs and catalog interfaces.

16. Final Stage Once all the hardware and software is ready for full- up testing, we should be ready to switch over to the final form of the hardware and software: A.New Heidenhain RCN8500 encoders can be installed. B.Final version of mount computer with all necessary network hardware configured, tested and installed. C.All-new telescope axis servo loops, interlock computer ready to replace, and can go to regular night-time use.

17. Team Roles Given the availability of the various people involved in the new mount system development, the team breaks down as follows* – PersonArea of Responsibility Dusty ClarkServo loops, system integration, hardware interfaces, telemetry services (mainly for servos), and interlocks. Keith PowellServo loop design and analysis assistance and upport for active vibration suppression development. Duane GibsonUser interfaces, catalog interface, message backbone, event logging and telemetry service back-end. Dallan PorterUser interface, catalogs, and network services (e.g. autoguiding and paddle inputs). Skip SchallerSystem OS administration, integration of EtherCAT library, astronomy code support, VxWorks code porting support. Richard Cool Joanna Hinz Validation of user and catalog interfaces, astronomy code testing, and support for instrument interfaces to the mount computer system. Tom Trebisky (emeritus member) Documentation, general support, and advice on porting code from the existing VxWorks system. * Considerable overlap among the various developers is expected, not all roles/responsibilities will necessarily be fixed.

18. Schedule This is a guess, based on where we are now, and given the bump in the schedule known as “aluminizing”: 4Q 2014: finish getting rotator-axis test computer and necessary EtherCAT interfaces and hardware ready to run servo loops. 1Q 2015: prepare new time service and astronomy code to bring the rotator-axis control operational equivalent to existing axis. Daytime operation only (night-time during M&E). 2Q 2015: Development suspended due to aluminization activities. 3Q-4Q 2015: Begin development of the other needed software infrastructure to begin main-axis controller operation. Develop main-axis control loops with existing encoders. 4Q 2015 – 2Q 2016: Prepare for arrival of new encoders, complete main- axis servo development, finish development of new user interfaces, publish/subscribe, and telemetry back-end. 3Q 2016: Final development and deployment of new encoders. Begin regular night-time operations on the new system.