1. Hall A meeting October 15 th 2013 Alexandre Camsonne

2. Outline Single board computers Hall A servers implementation Differences with VxWorks Read out code Adapting code Available codes Performance To do list Conclusion

3. Single board computers Core i7 single board computer 2.53 GHz 4 GB RAM Full Linux PC on a board VME320 XVB601/XVB602 Compact Flash XVR14 On board Flash

4. Hall A server implementation DHCP server PXE/TFTP and NFS server Accessible from adaql1 in /root/diskless/ If need to add a new CPU : need to know MAC address and add to DHCP and create directory for system files on CHAFS host intelha7 { filename "linux-install/pxelinux.0"; # File location relative to /tftpboot/ next-server 129.57.99.58; # TFTP server hardware ethernet 00:20:fa:fa:fa:fa; # Client MAC XVR14 fixed-address 129.57.255.255; # Client IP }

5. Difference with vxWorks CPUs Boot using PXE or onboard drive Centos 5.8 distribution No boot parameters, use DHCP to assign IP address Little Endian Use C directly instead of CRL Use virtual memory

6. Readout code CODA library : jvme Readout is C program with includes and link to JVME Supported and developped by Bryan Moffit https://coda.jlab.org/wiki/index.php/Intel- based_VME_Controllers /u/site/coda/3.0/linuxvme /u/site/coda/3.0/linuxvme/examples

7. Adapting programs vxWorksLinuxNotes STATUS sysBusToLocalAdrs (int adrsSpace, char *busAdrs, char **pLocalAdrs); int vmeBusToLocalAdrs(int vmeAdrsSpace, char *vmeBusAdrs, char **pLocalAdrs); int vxMemProbe(char *addr, int mode, int size, char *rval); int vmeMemProbe(char *addr, int size, char *rval);Linux, mode is read only STATUS intConnect (VOIDFUNCPTR *vector, VOIDFUNCPTR routine, int parameter); int vmeIntConnect(unsigned int vector, unsigned int level, VOIDFUNCPTR routine, unsigned int arg); int intDisconnect(int vector)int vmeIntDisconnect(unsigned int level);vector != level int sysIntEnable(int);NONE int sysIntDisable(int);NONE int intLock();NONE void intUnlock(int key);NONE STATUS usrVmeDmaConfig(unsigned int addrType, unsigned int dataType, unsigned int sstMode); int vmeDmaConfig(unsigned int addrType, unsigned int dataType, unsigned int sstMode); STATUS sysVmeDmaSend (UINT32 locAdrs, UINT32 vmeAdrs, int size, BOOL toVme); int vmeDmaSend(unsigned int locAdrs, unsigned int vmeAdrs, int size); Linux, always from VME int sysVmeDmaDone(int pcnt, int pflag);int vmeDmaDone(); vxWorks: returns size - (#bytes transferred) Linux: returns (#bytes transferred) https://coda.jlab.org/wiki/index.php/VxWorks_to_Linux_HOWTO

8. Readout code ( vme_list.c ) /************************************************************************* * * vme_list.c - Library of routines for readout and buffering of * events using a JLAB Trigger Interface (TI) with * a Linux VME controller. * */ /* Event Buffer definitions */ #define MAX_EVENT_POOL 400 #define MAX_EVENT_LENGTH 1024*10 /* Size in Bytes */ /* Define Interrupt source and address */ #define TIR_SOURCE #define TIR_ADDR 0x0ed0 /* TIR_MODE: 0 : interrupt on trigger, 1 : interrupt from Trigger Supervisor signal 2 : polling for trigger 3 : polling for Trigger Supervisor signal */ #define TIR_MODE 2 #include "linuxvme_list.c" /* source required for CODA */ #include "usrstrutils.c" /* helper routines to pass data from mSQL to ROC */ /* Globals to be filled by usrstrutils */ int string1=0, string2=0; /* defined when string is present in CODA database */ int ps1=0,ps2=0,ps3=0,ps4=0,ps5=0,ps6=0,ps7=0,ps8=0; /* defined in "ffile" */ /* function prototype */ void rocTrigger(int arg); void rocDownload() { /* Setup Address and data modes for DMA transfers * * vmeDmaConfig(addrType, dataType, sstMode); * * addrType = 0 (A16) 1 (A24) 2 (A32) * dataType = 0 (D16) 1 (D32) 2 (BLK32) 3 (MBLK) 4 (2eVME) 5 (2eSST) * sstMode = 0 (SST160) 1 (SST267) 2 (SST320) */ vmeDmaConfig(2,5,1); /* Initialize usrstrutils */ init_strings(); string1 = getflag("string"); string2 = getflag("string2"); printf("usrstrutils configuration:\n"); printf("\tstring1 = %d\n\tstring2 = %d\n",string1,string2); printf("rocDownload: User Download Executed\n"); } void rocPrestart() { unsigned short iflag; int stat; /* Check ffile for changes (usrstrutils) Useful for items that may change without re-download (e.g. prescale factors) */ init_strings(); ps1 = getint("ps1"); printf("usrstrutils configuration:\n"); printf("\tps1 = %d\n",ps1); /* Program/Init VME Modules Here */ printf("rocPrestart: User Prestart Executed\n"); } void rocGo() { /* Enable modules, if needed, here */ /* Interrupts/Polling enabled after conclusion of rocGo() */ } void rocEnd() { printf("rocEnd: Ended after %d events\n",tirGetIntCount()); } void rocTrigger(int arg) { int ii; /* Example: Raise the 0th (1<<0) and 2nd (1<<2) output level on the TI */ tirIntOutput(1<<0 | 1<<2); /* Insert some data here - Make sure bytes are ordered little-endian (LSWAP)*/ *dma_dabufp++ = LSWAP(0xda000022); for(ii=0; ii<20; ii++) { *dma_dabufp++ = LSWAP(ii); } *dma_dabufp++ = LSWAP(0xda0000ff); /* Drop all output levels on the TI */ tirIntOutput(0); }

9. Readout code ( vme_list.c ) /* Initialize usrstrutils */ init_strings(); string1 = getflag("string"); string2 = getflag("string2"); printf("usrstrutils configuration:\n"); printf("\tstring1 = %d\n\tstring2 = %d\n",string1,string2); printf("rocDownload: User Download Executed\n"); } void rocPrestart() { unsigned short iflag; int stat; /* Check ffile for changes (usrstrutils) Useful for items that may change without re-download (e.g. prescale factors) */ init_strings(); ps1 = getint("ps1"); printf("usrstrutils configuration:\n"); printf("\tps1 = %d\n",ps1); /* Program/Init VME Modules Here */ printf("rocPrestart: User Prestart Executed\n"); } void rocGo() { /* Enable modules, if needed, here */ /* Interrupts/Polling enabled after conclusion of rocGo() */ } void rocEnd() { printf("rocEnd: Ended after %d events\n",tirGetIntCount()); } void rocTrigger(int arg) { int ii; /* Example: Raise the 0th (1<<0) and 2nd (1<<2) output level on the TI */ tirIntOutput(1<<0 | 1<<2); /* Insert some data here - Make sure bytes are ordered little-endian (LSWAP)*/ *dma_dabufp++ = LSWAP(0xda000022); for(ii=0; ii<20; ii++) { *dma_dabufp++ = LSWAP(ii); } *dma_dabufp++ = LSWAP(0xda0000ff); /* Drop all output levels on the TI */ tirIntOutput(0); }

10. Linux VME drivers and readout JLAB boards : FADC250, F1,TI, TS CAEN V792, V775, V1190 Fastbus SFI 1877, 1881 Scaler SIS3801 MPD library ( C++ based )

11. Performances Most gain when transferring a lot of data Should benefit DVCS and experiment using FADC Up to 115 Mbytes/s, close to Gigabit ethernet line speed Twice faster than vxWorks CPU thanks to multicore

12. To do list Add adaq and a-onl directory Automatic start of ROC in startcoda, ROC not started automatically for debugging DVCS port of ARS and trigger libraries and debug Test INFN APV25 Readout with CODA Setup serial port and output to portserver

13. Conclusion New Intel VME CPU deployed Different way to boot Server infrastructure in place Readout available for most boards Board about twice more faster for about same price as Power PC