Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe

Joint Institute for VLBI in Europe Very Long Baseline Interferometry Very Long Baseline Interferometry

Ever larger telescopes 15cm 5m 100m

Electromagnetic radiation

(m) D (m) /D (rad) smallest optical biggest radio wavelengthdiameterpixelsize

A neat trick 100m

A neat trick 100m 20m  correlator

10,000km

A different ball game! (m) D (m) /D (rad) largest optical (IR) VLBI (radio)

Science

Recording the data Record up to 4096 Mbps (512 Mbyte/s)

Number of combinations to be computed as function of n ~n 2

16 telescopes at 1024 Mbps

UniBoard 4 frontnode 4 backnode 1Gbit/s ethernet 16x 10Gbps >20 Gbps/link

Erlang?! typical languages used –80% C++ –15% Python –PHP, Fortran, perl database –MySQL needed to convince management –pilot projects!

Decoding data files VLBI has framed data format(s) Look for ‘syncword’ –different flavours, same syncword, different framesize Other format uses different syncword

(not quite) surprising little code 212 lines of code –first dabblings... about a week’s work recognizes + decodes 3 formats decodes the BCD timestamps supports distributed system –reading file on one machine –decoding on other machine management duely impressed –got permission to do a larger pilot!

Soft real time monitoring website webcams at stations –webpage with http links = #FAIL –unreliable links really high latency (China, Chile) don’t want broken images –low bandwidth cache in Europe: multiple users eat all bandwidth parse telescope logfiles –immediate antenna calibration info –generate timeseries plots as observation progresses stored in MySQL

Soft real time monitoring website webcam URLs + status stored in MySQL –supervisor starts webcam monitors one gen_server for each webcam –loads info from MySQL –uses –strict checking on return value only HTTP 200 status replies content_length header set? explode if problem –use a port to ImageMagick to scale image –write binary JPEG data into mnesia

Soft real time monitoring website monitor all participating telescopes telescope log file (ASCII text) –retrieved via python + ssh –read into Erlang using a port –low data rate after start up parses lines –update MySQL generate plots using gnuplot –for each telescope –query MySQL every so many minutes –using a port –write. png binary data to mnesia

Soft real time monitoring website patch it all together using webtool cache complete webpage in mnesia –too old? generate new one serve.png and.jpg data from mnesia –image URL points back to webtool extremely simple to cache + serve data –brilliant idea: mapping URL to erlang fn call! in total used ~3000 lines of code –webcam, logfile monitoring; plotting; serving web pages –excellent test case for OTP and distributed system got the green light for the real thing!

UniBoard EthernetSwitch FPGA Nios2 CPU 1Gbit PHY FPGA Nios2 CPU 1Gbit PHY FPGA Nios2 CPU 1Gbit PHY FPGA Nios2 CPU 1Gbit PHY Control Computer windows/unix EthernetSwitch VHDL Hardware Software

FPGA 1Gbit PHY VHDL Hardware Software Polyphase Filterbank registers Delay Module registers Nios2 CPU registers

VHDL Hardware Software Control Computer windows/unix Nios2 CPU Client Server UDP/IPv4 over ethernet

Simple binary protocol Read N 32bit words starting from START ADDRESS (Over)Write N 32bit words from the packet to START ADDRESS Read/Modify/Write N 32bit words from START ADDRESS + packet to START ADDRESS

FPGA client library Registermap: list of register definitions type of the register (details follow) name of the register (symbolic) address of the register in the memory map one controller  one fpga

FPGA client library fpga:bit(,, ) fpga:bitrange(,,, ) fpga:word(, ) fpga:range(,, ) Available registerdefinition types for the controller

FPGA client library fpga:read( ) fpga:write(, ) fpga:or_(, ) fpga:and_(, ) fpga:xor_(, ) Available register commands

Example -module(firfilter). % this module is called firfilter, driving one of those -behaviour(personality). -export([registers/0, start_filter/2]). % this module implements an FPGA personality % Called automatically by the FPGA control framework registers() -> {ok, [ fpga:bitrange(num_tap, 5, 3, 16#24), fpga:bit(pps_enable, 28, 16#24), fpga:word(control_status, 16#abcc), ]. % define a high-level command for this personality start_filter(FPGA, NumTap) -> % disable the PPS fpga:execute(FPGA, fpga:write(pps_enable, 0)), % read the control status register case fpga:execute(FPGA, fpga:read(control_status)) of 1 -> fpga:execute(FPGA, fpga:xor(control_status, 42)); _ -> ok end, % however, you can easily execute a number of commands in one go: fpga:execute(FPGA, [fpga:write(num_tap, NumTap), fpga:write(pps_enable, 1)]).

Safety 2> fpga:execute(FPGA, fpga:write(pps_enable, 2)). OH NOES! error:"Value '2' does not fit in one bit (pps_enable)" FPGAControl terminating [{{badmatch, {reply, {error, {"Value '2' does not fit in one bit (pps_enable)", [{fpga,err,2, [{fil

Full correlator control system besides single fpga control –parallel control of eight fpga’s parsing + validating configuration files –using yecc decoding + sending over UDP of binary data files –data as well as polynomial coefficients capture network data to file configure and control the hardware MySQL read/write two weeks VHDL coding = < 1 day Erlang coding

data reader data writer UniBoard command/control Full correlator control system

The good cross platform –we have Linux, Mac OSX and Solaris distributed –typically many machines involved non-variability of ‘variables’ conciseness / amount of work per LOC connects to MySQL –not easy to connect other languages to mnesia hard crash on errors –about the most valuable property: can’t ignore! binary pattern matching –every language should be jealous of this re-use of development/testing code –it’s so easy to re-use code!

The bad our VHDL engineers struggle –language simple, but –concepts like recursion + distribution are ‘hard’ conciseness / amount of work per LOC –takes effort to read code written by someone who’s better at it than you deploying a developing distributed system –we haven’t found a simple way yet –dynamic remote code loading could be awsome it works for individual modules useless if you have dynamic dependencies... –erlang:apply/3

Thank you for your attention!