1. The Mark 5B VLBI Data System Alan Whitney for the Mark 5 team MIT Haystack Observatory 20 September 2006 5 th e-VLBI Workshop Haystack Observatory

2. The Mark 5 team Will Aldrich John Ball Peter Bolis Roger Cappallo Kevin Dudevoir Brian Fanous Dave Fields Chet Ruszczyk Jason SooHoo Mike Titus Dan Smythe Alan Whitney Ken Wilson

3. Mark 5A Data Acquisition System (introduced 2002)

4. Mark 5A Characteristics Direct plug-compatible replacement for 64-track Mark4 or VLBA tape drives Record/Playback at rates up to 1024 Mbps Accept 2 independent ‘8-pack’ disk modules can be used in ‘ping-pong’ fashion for nearly continuous recording Records 8, 16, 32 or 64 tracks from Mark4 formatter (1024 Mbps max) or VLBA formatter (512 Mbps max)

5. Current Mark 5A Status ~150 Mark 5 systems deployed; >95% of tape systems have been replaced >1200 Mark 5A ‘8-pack’ disk modules deployed (>2 PB of storage!); growing rapidly! Many stations are now entirely Mark 5A, except in some cases for data which goes to VLBA correlator 1 Gbps experiments are now routine for both geodesy and astronomy VLBA has mostly converted to Mark 5; all VLBA stations have Mark 5A, but correlator not yet fully outfitted

6. Mark 5B Characteristics Full VSI-H compatibility Same chassis as Mark 5A; uses same disk modules; requires Mark 5B I/O card 1024 Mbps record/playback Eliminates need for external formatters, but requires a VSI-H interface Station Unit capabilities for connection to Mark 4 correlators is designed into Mark 5B Extensive built-in phase-cal extraction and state counting on both data record and data playback Front-panel status display – 8 tri-color LEDs DIM and DOM capabilities are separate FPGA downloads FPGA is programmable via software Development support from Mark 5 development consortium – BKG, EVN, KVN, JPL, MPI, NASA, NRAO, USNO

7. VSI-H VSI-S VSI-E

8. Mark 5A/B System Comparisons Mk5AMk5B Data Interface Emulates Mk4/VLBA tape transport VSI-H (64MHz max clock rate) Max data rate 1024 Mbps Record modes 8, 16, 32, 64 “tracks” 1,2,4,8,16,32 bitstreams Disks Mk5 “8-pack”Same Chassis Mk5Same I/O card Mk5AMk5B SS card XF2 I/O-SS intf Modified FPDPFPDP

9. Mark 5B Functional Block Diagram

10. Mark 5B I/O Board FPDPVSI OutputVSI Cntl/Mntr VSI Input 256MB memory PCI Connector

11. Mark 5B Status Checkout of Mark 5B is complete; first rev of DIM software has been released Mark 5B is in regular use at Westford antenna; also used in large mm experiment in April 2006 Mark 5B is interfaced to Haystack Mark 4 correlator and in routine production use ~30 Mark 5B I/O boards have been built and tested – will be distributed to Mark 5 development consortia members in near future Mark 5B can be ordered from Conduant Corp in Loveland, CO

12. Reasons to adopt Mark 5B Eliminate need for expensive external formatters; particularly important for new stations or stations without existing Mark 4 or VLBA formatters DBEs and dBBCs will require VSI-H interface With a 14-BBC Mark4, up to 1792 Mbps can be recorded with two parallel Mark 5B systems; current systems can only generate a maximum of 1024 Mbps of formatted data due to Mk4 formatter limitation –requires modification of Mark 4 formatter for VSI-H Extensive phase-cal extraction and state counting capabilities for better diagnostics and better system calibration Replace unreliable Station Units on Mark 4 correlators; SU capability is built into Mark 5B

13. Mark 5A/B Compatibility Mark 5B can play only Mark 5B recordings (VSI format in/out) Upgraded Mark 5A (“Mark 5A+”) can play: -All Mark 5A recordings -Mark 5B recordings made in almost all modes; playback is in VLBA-track-format Mark 5A+ design is complete and tested, but is needed only on correlators that do not yet support Mark 5B Existing Mark 5A systems can be upgraded to Mark 5A+ with new Xilinx download and upgraded software

14. Mark 5B+ (2048Mbps) Conduant has introduced an upgraded StreamStor (dubbed “Amazon”) that supports up to ~3 Gbps on FPDP2 interface Mark 5B I/O card has been designed to support input VSI-H clock rate of 64MHz, as well as FPDP2 compatibility, to support max recording rate of 2048 Mbps with Amazon board May be desirable to record across 2 disk modules (16 disks) simultaneously Playback is limited to 1024 Mbps Recordings made on Amazon will be playable on standard Mark 5B system No promises yet on date of general availability, though hope in a few months

15. Mark 5A/B/B+ System Comparisons Mk5AMk5BMk5B+ Data Interface Emulates Mk4/VLBA tape transport VSI-H (64MHz max clock rate) Max data rate 1024 Mbps 2048 Mbps Record modes 8, 16, 32, 64 “tracks” 1,2,4,8,16,32 bitstreams Same as Mk5B Disks Mk5 “8-pack”Same Chassis Mk5Same I/O card Mk5AMk5B SS card XF2 Amazon I/O-SS intf Modified FPDPFPDPFPDP2 (clocks on both edges

16. Mark 5 Upgrade Costs Target Existing Mk5AMk5BMk5B+ 0$18K~$20K~$23K Mk5A-~$3K (Mk5B I/O) ~$12.5K (Mk5B I/O plus Amazon) Mk5B--~$9.5K (Amazon) Note: Does not include external cabling costs, typically a few hundred dollars

17. Mark 5B Software Current Mark 5B DIM software is built into same program as Mark 5A, but is renamed Mark5B New program called dimino is being written by Kevin Dudevoir exclusively for Mark 5B DIM support –includes advanced network interface management and VSI-E support –includes phase-cal processing support Mark 5B DOM support on Haystack/MPI/USNO correlators is separate program called domino

18. Disk-Media Status Hard disk price vs capacity/performance continues to drop -Now at or below ~$0.50/GB and continues to drop (Mark 4/VLBA tape was ~$2.00/GB) 400 GB disks are becoming more affordable – 8-pack of 400GB disks comparable to ~11 VLBA/Mark 4 tapes 750 GB disks are now available – 8-pack of 750GB disks comparable to ~21 VLBA/Mark 4 tapes; ~26 hours @ 1 Gbps unattended! 1 TB disks will soon be available

19. Tape vs. Disc Price Comparison

20. Disk-Media Reliability Based on statistics collected at Haystack, average disk drive failure rate is ~0.5% per year Failure rate of Hitachi 250GB has been higher than average, but may now have been fixed Conduant is shipping these disks in assembled Mark 5 disk modules Disk reliability at high was investigated in Mark 5A tests on Mauna Kea in early 2006: Tested disk drive types were: Maxtor 300-GB Model 7L300R0 Seagate 300-GB Model ST3300831A Western Digital 320-GB Model WD3200SB-01KMA0 Hitachi 250-GB Model HDS722525VLAT80 →Only the Hitachi’s sustained good performance at 14000ft (However, all disks recovered performance when returned to low altitude)

21. Plans for Serial-ATA Support A new module is being designed to support SATA disks Existing 200-pin connector on module will be maintained using PATA signals Conversion to SATA will be on module backplane Prototype SATA modules should be ready in next few months, but we don’t feel urgency at present time

22. Mark 5B Interface Boards VSI-4 sampler adapter board Will be placed inside Mark 4 formatter to upgrade to VSI Uses existing Mark 4 samplers and 1pps generator 2 VSI output connecters at 1024Mbps each (though data rate is restricted to 1792 Mbps by 14 BBC’s) Prototype boards ready for checkout VSI-C sampler adapter Interfaces VLBA samplers to VSI Designed at Metsahovi Correlator Interface Board (CIB) Interface between Mark 5B and Mark 4 correlator PCB currently in fabrication Upgraded Serial Link boards for Mark 4 correlator Designed at MPI Prototypes have been tested; replication to begin soon

23. Further development of Mark 5 Considering direct 10GigE interface into StreamStor board to support recording up to 4Gbps Considering direct 10GigE data interface from DBE at 4Gbps

24. VLBI Standard Interface (VSI) Joint effort by astronomy (GVWG) and geodetic (IVS) communities The purpose of VSI is to define a standard interface to and from a VLBI Data Transmission System (DTS) that allows heterogeneous DTS’s to be interfaced to both data-acquisition and correlator systems with a minimum of effort Focuses on those functions independent of DTS technology VSI-H – complete (Aug 2000) Data and control interfaces for recording and playback, including connectors and pinouts Electrical and timing specs Won national award in Japan in 2001! VSI-S – complete (Feb 2003) Communications model Application Protocols Command/Response Syntax Suggested base command set VSI-E (e-VLBI) – draft specification and reference implementation available VSI specs available at http://www.haystack.edu/vsihttp://www.haystack.edu/vsi VSI is now widely accepted worldwide with a substantial number of VSI-H compliant components available

25. Mark 4 Station Unit emulation Mark 4 Station Unit