1. WP8: FlexBuff Ari Mujunen, Tomi Salminen Aalto University Metsähovi Radio Observatory NEXPReS 2nd Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark Research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007- 2013) under grant agreement n° RI-261525. This presentation reflects only the author's views. The European Union is not liable for any use that may be made of the information contained therein.

2. 2 WP8: FlexBuff  The full title in the “Description of Work” is really, really long:  “WP8: Provisioning High-Bandwidth, High-Capacity Networked Storage on Demand”  Thus introducing a new acronym for these “flexible buffers”  Before the unofficial one “AriBox” gets adopted irrevocably... NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark

3. 3 WP8 Objective  Determine the best practical mix of solutions  What kind of storage, where, how connected, how allocated/deallocated and accessed  Which will serve the needs of evolving (>1Gbps) VLBI data acquisition and processing NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark

4. 4 Primary Objective  Be able to reliably record a local high-speed UDP packet stream onto local buffer disks  dBBC/FiLa10G, Roach1/2 variants, iBOB...  Allow simultaneous long-distance remote read access, for correlation processing  Long-distance & high-speed implies UDP NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark

5. 5 Partners  Focus areas  AALTO Coordination, basic technologies  ASTRON Long-term archival & reprocessing  INAF Global/local allocation/deallocation schemes  JIVE Augmenting correlation capabilities with buffering  OSO Trial-site performance & applicability testing  PSNC Role & trials of computing center buffering  UMAN Trial-site performance & applicability testing NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark

6. 6 Overview of P2 of WP8  Implemented “vlbi-streamer” software  After a notable delay  Testing started, good initial results  Completed deliverables  Delayed deliverables  Meetings and other activities  P1 Review recommendations NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark

7. 7 vlbi-streamer  Available at:  http://code.google.com/p/vlbi-streamer/  Recording local UDP streams  Concurrent interleaved reading of the same disks  Shows that initial architectural decisions were correct by its performance  Large enough direct/asynch I/O blocks per each disk NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark

8. 8 Recent vlbi-streamer Test Results  Local UDP streaming performance tests  Wirespeed 10GE  Long (30min) tests show writing at max wire speed and 0 ploss  Writing to 34 disks /wo net  Architecture can handle 40Gbps; always >30Gbps. Close to HW- limit.  Using INAF’s hw RAID system (/wo net)  Works reasonably well even there, 10-14Gbps. Lower performance is due to HD access granularity (~32MB writes). Speed increased by 4Gbps with larger writes (256MB-512MB).  Long-distance UDP streaming tests  NORDUnet does not have a lightpath up anymore, so Jb Mh tests on hold NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark

9. 9 Recent Haystack Developments: Mark6  Essentially the same hardware & Linux except:  Except hard disks are in external enclosures  Physically nearly identical to old Mark5 8-disk packs  Two external MiniSAS connectors added to the front panel  Same software can run on WP8 & Mark6 hw  Current Mark6 software is Mark5C-style raw Ethernet packet capture onto raid0 volumes  http://vdas.org/  And WP8 hw can have controllers with ext MiniSAS NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark

10. 10 FlexBuff / Mark6  Just different “packaging” of disks  Fixed vs. shippable NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark 5U, 36 disks Mark6 figures: “Mark6 data system DiFX presentation”, A.Whitney, D.Lapsley, 2011.12.05, http://www.haystack.edu/tech/vlbi/mark6/mark6_memos/04-2011.12.05_Mark6_data_system-DiFX_mtg-Haystack.pdf

11. 11 P2 Completed Deliverables  D8.3 (INAF) was overdue, delivered on 29-Sep-2011  “Design document of storage element allocation methods”  D8.4 (AALTO) was due, was a bit late, but delivered on 02-May-2012  “Design document of transparent local/remote application programming interface for storage elements” NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark

12. 12 P2 Delayed Deliverables  These were due before end of Period 2:  D8.05 (OSO) month 21  “Performance and integration test report of simultaneous I/O storage elements”  D8.06 (INAF) month 22  “Test report of storage element allocation methods”  D8.07 (UMAN) month 22  “Test report of transparent local/remote application programming interface for storage elements”  Delayed, since the tests on which they are based are delayed  This was mainly caused by the >6 month interruption in vlbi- streamer development at AALTO due to lack of manpower and problems in recruiting NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark

13. 13 Catching up with P2 Delayed Delivs NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark  Move test work +6 months  More overlap with integration work

14. 14 P1 Review Recommendations  The review panel gave one recommendation to WP8:  “R6: The future deliverables in WP8 must describe explicitly in which way the technology choices are influenced (or not...) by the user requirements identified (e.g. D7.01 and D7.02).”  Steps taken:  WP7 deliverables studied and findings being integrated into vlbi-streamer  WP7 participation in WP8 meetings & telecons NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark

15. 15 P2 Meetings and Other Activities  Face-2-Face Meeting @Schiphol  Mon 23 April 2012  Several telecons in 2Q2012 NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark

16. 16  Mon 23 April 2012  FlexBuff specifications refined  vlbi-streamer and jive5ab integration discussed  Distribution of tasks among partners Face-2-Face Meeting @Schiphol NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark WP 8 Photo: T. Charles Yun

17. 17 Overview of WP8 Final Period P3  Remaining deliverables  Upcoming meetings and activities  Discussion on timing and schedule NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark

18. 18 P3 Remaining Deliverables  In addition to delayed test reports  D8.05, D8.06, D8.07  The deliverables dealing with correlation testing  D8.08 (JIVE) month 26  “Design document of integration system of allocation methods with transparent multi-Gbps access”  D8.09 (ASTRON) month 32  “Integration test report of LTA and pipeline integration”  D8.10 (JIVE) month 32  “Demonstration tests of the integrated system”  D8.11 (AALTO) month 35  “Final report on Networked Storage on Demand” NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark

19. 19 WP8 Upcoming Meetings and Activities  2 nd Face-2-Face meeting in conjunction with EVN TOG meeting in Onsala 26-Jun-2012  Plans for “testing workshops”  In close cooperation with WP7 sw correlation tests  Some of the planning for the WP8 meetings depends on the Board agreeing to support WP7&WP8 with travel funds NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark

20. 20 WP8 Schedule NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark

21. 21 Discussion on Timing and Schedule  Field System integration started  Script for adding commands to snp-files. No feedback from recorder to field system, but deeper field system integration deemed a timesink.  Real data tests approaching  With jive5ab as data source  With WP7 software correlator NEXPReS Second Board Meeting – 2012 June 20 – NORDUnet, Copenhagen Denmark