1. www.egi.eu EGI-InSPIRE RI-261323 EGI-InSPIRE www.egi.eu EGI-InSPIRE RI-261323 GPGPU Accounting John Gordon STFC 09/04/2013 EGI CF – Accounting and Billing1

2. www.egi.eu EGI-InSPIRE RI-261323 Accounting Overview Parse Batch Logs Get Identity Information Join and cut UR Publish to APEL APEL receives UR into database APEL Summarises Portal Visualises from Summaries 20/09/2011 Business Models & Sustainability – EGI TF Lyon - Sept 2011 2

3. www.egi.eu EGI-InSPIRE RI-261323 Parse Batch Logs Today for CPU Accounting we support –LSF –GE –Torque –SLURM (in EMI 3.1) Do these batch systems report GPUs in their batch logs? –Lots of caveats. Still to be fully confirmed. 20/09/2011 Business Models & Sustainability – EGI TF Lyon - Sept 2011 3

4. www.egi.eu EGI-InSPIRE RI-261323 Get Identity Information No change Done by BLAH in CREAM 20/09/2011 Business Models & Sustainability – EGI TF Lyon - Sept 2011 4

5. www.egi.eu EGI-InSPIRE RI-261323 Join and cut UR Extend UR to include second type of CPU –Do this in conjunction with OGF UR-WG. –UR 2.0 doesn’t support multiple CPU types in a job or attached processors. –nb Intel Phi is an attached processor(s) so we will need to address this anyway. 20/09/2011 Business Models & Sustainability – EGI TF Lyon - Sept 2011 5

6. www.egi.eu EGI-InSPIRE RI-261323 Publish to APEL SSM ignores the content of the records it transports 20/09/2011 Business Models & Sustainability – EGI TF Lyon - Sept 2011 6

7. www.egi.eu EGI-InSPIRE RI-261323 APEL receives UR into database APEL would need to extend the job record database to add extra columns Simple for a test system but a mammoth task to migrate the production database. 20/09/2011 Business Models & Sustainability – EGI TF Lyon - Sept 2011 7

8. www.egi.eu EGI-InSPIRE RI-261323 APEL Summarises Aggregating total GPU Use for a User, VO, Site, Country, NGI, etc is straightforward Correlations like GPU/CPU, GPU/wallclock, etc are more complicated –Requirements? 20/09/2011 Business Models & Sustainability – EGI TF Lyon - Sept 2011 8

9. www.egi.eu EGI-InSPIRE RI-261323 Portal Visualises from Summaries Straightforward to add other variables GPU, normalised GPU to the existing ones. Portal displays one variable (njobs, totcpu, etc) as function of two others –Site vs VO –Region vs Date –Etc. 20/09/2011 Business Models & Sustainability – EGI TF Lyon - Sept 2011 9

10. www.egi.eu EGI-InSPIRE RI-261323 Summary Most things possible once the major condition is met. i.e. Batch logs report reliably and understandably on GPU identified with a batch job Still a lot of work to be done to understand this See John Walsh’s experiences PRACE reported that they don’t. 20/09/2011 Business Models & Sustainability – EGI TF Lyon - Sept 2011 10

11. www.egi.eu EGI-InSPIRE RI-261323 What Can We Do? The APEL Team could implement a test server receiving records from a hacked apel client once the research had been done on retrieving the relevant infor from batch records After the end of EMI and with a full roadmap for 2013 we don’t have enough effort to do this research or to do the development in extending the relevant parsers for all batch systems. 20/09/2011 Business Models & Sustainability – EGI TF Lyon - Sept 2011 11